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The  purpose  of  this  study  was  to  develop  a  rating  scale 
that  structures  clinicians'  evaluations  of  a  newborn's  risk 
of  future  maltreatment;  compared  to  unstructured  judgments, 
such  an  instrument  should  improve  the  ability  of  clinicians 
to  recognize  those  patients  who  would  benefit  most  from 
scarce  social  services.  The  Clinical  Rating  Scale  (CRS)  was 
composed  of  22  risk  factors  for  child  maltreatment.  Each 
item  was  rated  on  a  four-point  dichotomous-ordinal  scale. 

On  the  CRS,  a  clear  description  was  given  for  each  level  of 
risk  for  each  item.  The  CRS  yielded  a  binary  rating  of  ei¬ 
ther  High  or  Low  Risk.  Thirty-two  newborns  and  their  moth¬ 
ers  were  evaluated  by  pairs  of  experts  in  the  field  of  child 
maltreatment  using  their  unstructured  clinical  judgments  (of 
which  the  consensus  rating  served  as  the  gold  standard  of 
prediction)  and  then  using  the  CRS.  Each  child  and  his/her 
family  were  also  evaluated  by  the  nurse  and  pediatrician 
taking  care  of  them  during  their  post-partum  hospital  stay. 
As  part  of  a  longitudinal  predictive  validation  study, 
clinicians  using  the  CRS  evaluated  363  consecutive  newborns. 

The  agreements  in  risk  ratings  by  the  pairs  of  experts 
using  unstructured  judgments  (kappa=0.80)  and  the  CRS 
(kappa=0.65)  were  computed.  The  sensitivity  (SN)  and  speci¬ 
ficity  (SP)  of  the  CRS  when  used  by  the  experts  were  comput¬ 
ed  (SN=100%,  SP=  51%);  the  agreement  of  each  expert's  CRS 
rating  compared  to  his/her  own  unstructured  judgment  was 
computed  (kappa=0 . 45 ) .  The  sensitivity  and  specificity 
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of  the  predictions  by  the  non-experts  using  both  their  un¬ 
structured  judgments  ( SN  =40%,  SP  =100%;  SN  =63%, 

J  J  nurse  nurse  peds 

SP  =92%)  and  the  CRS  (SN  =64%,  SP  =78%;  SN  =63%, 

peds  nurse  nurse  peds 

SPpeds=92%)  were  computed.  Item  by  item  inter-rater  agreement 
from  the  longitudinal  data  using  weighted  kappa  showed  good 
inter-rater  agreement. 

The  CRS  structures  clinical  judgment  about  risk  of  mal¬ 
treatment,  improves  the  identification  by  non-expert  clini¬ 
cians  of  those  at  risk,  and  yields  good  inter-rater  agree¬ 
ment  in  judgments.  It  should  be  useful  to  clinicians  in 
identifying  children  and  families  at  high  risk  of  subsequent 
abuse  or  neglect. 


CHAPTER  ONE: 


INTRODUCTION 
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In  1962,  Henry  Kempe  and  his  colleagues  gave  the  first 
formal  medical  recognition  to  a  medical/social  problem  that 
has  afflicted  families  for  centuries.  Their  pioneering  work 
in  defining  the  "battered-child  syndrome"  laid  the  ground¬ 
work  for  the  acceptance  of  child  maltreatment  as  a  real 
problem  that  clinicians  must  be  able  to  recognize  and  to 
treat. 

The  article  by  Kempe  and  colleagues  triggered  intense 
research  interest  in  the  causes,  modes,  symptoms,  and  ef¬ 
fects  of  child  maltreatment.  While  the  first  article  fo¬ 
cused  on  identifying  those  children  who  had  already  suffered 
maltreatment,  the  question  soon  arose  whether  it  would  be 
possible  to  identify  those  children  who  are  as  yet  unharmed 
but  who  are  at  risk  of  being  abused.  Armed  with  the  ability 
to  recognize  such  children,  health  professionals  could  en¬ 
list  support  services  (such  as  parenting  classes,  social 
work  follow-up,  support  groups,  or  a  visiting  nurse  for  in- 
home  visits)  for  those  families  who  need  it  and  thus  prevent 
child  maltreatment  from  occurring. 

Over  twenty  years  of  work  by  many  different  researchers 
and  clinicians  has  produced  a  multitude  of  approaches  and 
instruments.  Some  methods  utilize  an  intense  psychosocial 
interview  of  parents  (or  parents-to-be)  while  others  rely  on 
a  check-list  of  socio-economic  issues  considered  to  be  risk 
factors.  Still  others  have  attempted  to  predict  maltreat¬ 
ment  using  only  unstructured  clinical  judgment  rooted  in 
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clinical  experience.  These  different  approaches  also  have 
produced  a  multitude  of  results  ranging  from  excellent  to 
poor  predictive  accuracy. 

In  order  for  a  predictive  instrument  to  be  useful  in 
the  routine  screening  of  families  it  must  be  non-threatening 
to  the  patients  and  fairly  easy  to  complete  with  high  pre¬ 
dictive  accuracy,  sensitivity,  and  specificity.  None  of  the 
instruments  previously  developed  fulfills  these  criteria. 
Those  with  excellent  statistical  indices  reguire  lengthy 
interviews  or  psychological  profiles,  sometimes  over  several 
visits,  which  make  them  unfeasible  for  routine  screening  and 
clinical  use.  Simpler  instruments  have  poor  statistical 
indices  because  of  reliance  on  items  not  highly  predictive 
of  child  maltreatment  and  omission  of  those  items  that  are 
highly  predictive.  The  reason  for  this  is  that  most  instru¬ 
ments  have  been  developed  for  research  purposes  with  goals 
other  than  routine  screening  in  mind. 

The  purpose  of  this  study  was  as  follows: 

1)  To  develop  a  predictive  instrument  designed  specifical¬ 
ly  to  screen  all  families  of  newborns  in  order  to  iden¬ 
tify  early  those  at  risk.  This  instrument  would  stand¬ 
ardize  clinical  judgment  about  parenting  ability  and 
would  be  convenient  enough  to  use  on  all  patients  dur¬ 
ing  the  postpartum  period. 

2)  To  refine  the  instrument  through  pilot  testing. 

3)  To  develop  a  scoring  method  for  the  instrument. 

4)  To  test  the  sensitivity  and  specificity  of  the  instru- 
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ment  compared  to  a  gold  standard  of  prediction  (concur¬ 
rent  validity). 

5)  To  examine  inter-rater  agreement  among  expert  clini¬ 
cians  . 

6)  To  assess  which  items  on  the  instrument  are  difficult 
for  non-experts  to  judge. 

Since  both  preventive  services  and  in-depth  psychologi¬ 
cal  evaluations  for  abuse  potential  are  expensive  and  diffi¬ 
cult  to  offer  to  an  entire  population,  this  new  instrument 
will  be  a  valuable  first-step  screening  instrument  that 
should  identify  those  families  in  need  of  the  more  intensive 
evaluations  and  preventive  services. 

Along  with  a  critical  review  of  past  research  on  risk 
factors  for  child  maltreatment  and  past  research  on  predic¬ 
tive  instruments,  this  thesis  gives  a  perspective  on  the 
problem  of  child  maltreatment  and  the  need  for  a  way  to 
identify  those  at  high  risk.  This  thesis  also  examines  the 
area  of  clinical  scales,  the  measurement  of  "soft"  data,  and 
the  problems  that  must  be  overcome  to  develop  such  an  in¬ 
strument.  These  topics  are  all  covered  in  Chapter  Two. 
Chapter  Three  details  the  methods  employed  in  conducting 
this  study.  Chapter  Four  contains  the  results,  and  Chapter 
Five  contains  a  discussion  of  the  results,  a  discussion  of 
the  ethical  issues  involved  in  a  routine  screening  program, 
and  a  proposal  for  the  minimum  criteria  that  a  screening 
instrument  must  meet  in  order  to  be  useful. 


CHAPTER  TWO 


BACKGROUND 
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An  Historical  Perspective  on  Child  Maltreatment 

Child  maltreatment  can  be  defined  as  any  form  of  child 
care  that  physically  or  psychologically  harms  a  child. 

Kempe  and  colleagues  defined  "the  battered  child  syndrome" 
in  1962  as  a  clinical  condition  (fractures,  bruises,  soft 
tissue  swelling,  f ailure-to-thrive ,  sudden  death)  that  re¬ 
sulted  from  physical  abuse  (Kempe  et  al.,  1962).  Maltreat¬ 
ment  goes  beyond  physical  abuse,  though.  Fifty  percent  of 
children  with  f ailure-to-thrive  have  no  organic  cause; 
theirs  is  a  problem  in  parenting  and  is  estimated  to  affect 
one  in  one  hundred  American  children  (Harris,  1982).  The 
effects  of  a  lack  of  sensory  or  social  stimulation  on  growth 
and  development  are  well  documented  in  institutionalized 
children.  The  effect  on  children  who  are  deprived  of  such 
stimulation  by  their  parents  is  the  same,  and  these  children 
can  experience  non-organic  f ailure-to-thrive  (Harris,  1982). 
In  1981,  the  Select  Panel  for  the  Promotion  of  Child  Health 
recognized  blows,  burns,  sexual  assault,  starvation,  con¬ 
finement,  exposure  to  unsafe  environments,  and  absence  of 
affection  or  attention  all  as  forms  of  child  maltreatment 
(U.S.  Dept.  HHS ,  1981).  Child  maltreatment,  then,  includes 
physical  or  verbal  violence,  sexual  exploitation,  neglect  of 
basic  needs,  and  abandonment. 

Acceptance  of  this  definition  is  predicated  on  an  ac¬ 
ceptance  of  children  as  people  possessing  human  rights. 
Historically,  children  have  not  been  viewed  as  such.  Nei- 
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ther  the  abandonment  of  children  in  ancient  Rome  (as  in  the 
story  of  Romulus  and  Remus),  nor  the  sacrificing  of  children 
in  Biblical  times  (as  in  the  stories  of  Abraham  and  Isaac, 
and  the  killing  of  male  babies  by  Herod)  were  viewed  as  hei¬ 
nous  crimes.  In  colonial  America,  it  was  considered  within 
parental  rights  to  whip,  castrate,  or  kill  misbehaving  chil¬ 
dren.  This  concept  is  reflected  in  the  adage  "spare  the  rod 
and  spoil  the  child"  (Straus,  Gelles,  Steinmetz,  1981). 

Legal  protection  for  children  from  their  parents  is  a 
recent  historical  development.  There  existed  in  the  United 
States  a  Society  for  the  Prevention  of  Cruelty  to  Animals 
before  there  was  a  Society  for  the  Prevention  of  Cruelty  to 
Children;  thus,  the  first  case  of  child  maltreatment  was 
prosecuted  in  1874  under  the  SPCA  laws  (Straus,  Gelles, 
Steinmetz,  1981).  It  took  the  Social  Security  Act  to  fund 
the  first  public  services  in  1935  "for  the  protection  and 
care  of  homeless,  dependent  and  neglected  children  and  chil¬ 
dren  in  danger  of  becoming  delinquents"  (U.S.  Dept.  HHS , 
1988).  It  was  ten  more  years  before  physicians  began  to 
consider  child  maltreatment  a  national  problem;  this  oc¬ 
curred  when  radiologists  started  to  notice  a  recurrent  pat¬ 
tern  of  healing  bone  that  was  characteristic  of  fractures 
resulting  from  intentional  blows  rather  than  accidental  in¬ 
jury.  It  was  the  work  of  Kempe  and  colleagues,  though,  that 
highlighted  and  clearly  defined  the  problem  in  1962,  stimu¬ 
lating  both  widespread  public  and  medical  concern  (Straus, 
Gelles,  Steinmetz,  1981).  From  1963  to  1966,  49  states  en- 
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acted  laws  requiring  the  reporting  of  suspected  cases  of 
child  abuse  or  neglect  to  a  designated  public  agency,  and  by 
the  end  of  the  1960's  all  50  states  had  such  laws  (U.S. 

Dept.  HHS,  1988;  Straus,  Gelles,  Steinmetz,  1981).  In  1974, 
the  Child  Abuse  Prevention  and  Treatment  Act  created  the 
National  Center  for  Child  Abuse  and  Neglect  (NCCAN)  to  "sup¬ 
port  state  and  local  efforts"  at  prevention  and  treatment 
(U.S.  Dept.  HHS,  1988). 

Without  recognition  of  child  maltreatment  as  a  problem, 
much  less  reporting  laws,  it  is  difficult  to  estimate  the 
extent  of  child  maltreatment  prior  to  the  mid-1960's.  It  is 
known  that  in  1968,  more  children  less  than  five  years  of 
age  died  from  parental  injuries  than  from  tuberculosis, 
whooping  cough,  polio,  measles,  diabetes,  rheumatic  fever, 
and  appendicitis  combined  (Straus,  Gelles,  Steinmetz,  1981). 
One  of  the  tasks  undertaken  by  the  NCCAN  was  to  clarify  the 
extent  of  the  problem.  The  NCCAN  initiated  a  Study  of  the 
National  Incidence  and  Prevalence  of  Child  Abuse  and  Neglect 
(a.k.a.  National  Incidence  Study  or  NIS).  The  original 
study,  which  was  completed  in  1980,  was  updated  in  a  second 
study,  completed  in  1986.  This  second  study,  NIS-2,  counted 
those  children  "who  experienced  demonstrable  harm  as  a  re¬ 
sult  of  maltreatment"  (the  core  estimate)  as  well  as  a  "sup¬ 
plementary  estimate"  of  children  endangered  (at  risk  but  not 
harmed  yet).  NIS-2  included  children  in  both  categories  who 
were  known  to  protective  services  or  any  third  party  (e.g. 
day  care  or  hospital  personnel)  (U.S.  Dept.  HHS,  1988). 


NIS-2  found  that  in  1986,  16.3  out  of  every  1000  chil¬ 
dren,  or  more  than  1.02  million  children,  experienced  "de¬ 
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monstrable  harm"  (the  core  estimate).  Fifty-six  percent 
were  in  the  form  of  abuse  and  forty-eight  percent  were  in 
the  form  of  neglect.  This  represented  a  66%  overall  in¬ 
crease  from  the  NIS-1  data  of  1980,  with  a  74%  increase  in 
abuse  alone,  and  a  200%  increase  in  sexual  abuse.  There  was 
no  change  in  neglect.  The  overall  fatality  rate  was  0.1% 
and  was  more  common  in  younger  children.  When  both  children 
harmed  and  children  endangered  were  counted  (core  estimate 
plus  supplemental  estimate),  NIS-2  reported  a  rate  of  25.2 
out  of  every  1000  children,  or  1.5  million  children  affect¬ 
ed.  That  translates  into  a  lifetime  prevalence  of  from  10 
to  40%  of  all  adults  who  experienced  some  form  of  maltreat¬ 
ment  as  a  child.  Sixty-three  percent  of  these  cases  in¬ 
volved  neglect  and  forty-three  percent  involved  abuse. 

NIS-2  also  examined  reporting  patterns  and  found  that 
noninvestigatory  agencies  (e.g.  schools,  hospitals)  discov¬ 
ered  more  than  five  times  the  number  of  cases  than  investi¬ 
gatory  agencies  (e.g.  police,  public  health  services); 
schools  reported  the  most  followed  by  hospitals  and  social 
service  agencies.  Of  those  children  who  had  actually  expe¬ 
rienced  harm,  only  40%  were  reported  to  child  protective 
services  (U.S.  Dept.  HHS ,  1988). 

The  American  Association  for  Protecting  Children  (AAPC) 
found  similar  results  for  1986.  The  AAPC  data  were  compiled 
from  reports  made  to  individual  state  child  protective  ser- 


vice  agencies.  They  reported  that  1.7  million,  or  32.8  out 
of  every  1000,  children  were  affected  involving  1.3  million 
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families.  This  represents  a  212%  increase  over  the  time 
period  1976  to  1986.  Of  the  reported  cases,  approximately 
40%  were  substantiated,  the  average  age  of  the  victim  was 
7.2  years,  52.5%  of  victims  were  female,  the  average  age  of 
the  perpetrator  was  31.7  years,  55.9%  of  the  perpetrators 
were  female,  and  48.9%  of  the  families  were  on  public  as¬ 
sistance.  The  racial  profile  of  maltreated  children  was 
essentially  parallel  to  that  of  all  children.  The  age  pro¬ 
file  of  maltreated  children  was  skewed  towards  younger  chil¬ 
dren;  43%  of  affected  children  were  aged  0-5  years  yet  this 
age  group  made  up  only  34%  of  all  children.  Children  aged 
6-11  years  made  up  31%  of  all  children  but  represented  33% 
of  the  maltreated  population  while  those  aged  12-17  made  up 
35%  of  the  population  but  only  24%  of  the  maltreated  popula¬ 
tion  (American  Association  for  Protecting  Children,  1988). 

The  extent  of  the  problem  and  the  steady  increase  in 
reported  cases  over  the  past  decade  can  be  attributed  in 
part  to  an  emphasis  on  community  awareness  of  the  problem 
and  thus,  increased  willingness  of  people  to  report  suspect¬ 
ed  cases  (U.S.  Dept.  HHS ,  1988).  However,  the  data  probably 
also  reflect  a  real  increase  in  child  maltreatment  that  is  a 
part  of  the  overall  national  problem  of  domestic  violence. 

It  is  estimated  that  on  average  yearly  in  America,  in  one 
out  of  six  households  one  spouse  strikes  the  other  spouse. 

In  three  out  of  five  households  with  children  at  least  one 
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parent  strikes  a  child.  In  three  out  of  five  households 
with  more  than  one  child,  there  is  violence  among  the  chil¬ 
dren.  Overall,  half  of  all  households  experience  some  form 
of  domestic  violence  once  a  year.  During  the  entire  course 
of  a  marriage,  in  more  than  one  out  of  four  couples  one 
partner  will  strike  the  other  (Straus,  Gelles,  Steinmetz, 
1981;  Straus,  Gelles,  1986). 

In  outlining  its  Objectives  for  the  Nation  Concerning 
the  Promotion  of  Health/Prevention  of  Disease  in  1980,  the 
U.S.  Department  of  Health  and  Human  Services  ranked  control 
of  violent  behavior  as  one  of  14  priority  areas.  Ten  spe¬ 
cific  objectives  for  control  of  violence  were  later  outlined 
and  included: 

By  1990,  the  proportion  of  the  primary  care  physicians 
who  take  a  careful  history  related  to  personal  stress 
and  psychological  coping  skills  should  be  more  that 
60%.... By  1990,  injuries  and  deaths  to  children  in¬ 
flicted  by  abusing  parents  should  be  reduced  by  at 
least  25%  (Silver,  Goldston,  Silver,  1984). 


The  Role  of  Predict ion /Prevent ion 

Child  maltreatment  in  the  form  of  physical  abuse  is 
clearly  a  danger  to  the  child's  life,  growth,  and  develop¬ 
ment.  However,  all  forms  of  maltreatment  threaten  a  child's 
well-being.  Neglect  can  lead  to  death  from  accidents  or 
exposure  to  harmful  elements.  Non-organic  failure-to-thrive 
in  infancy  and  psychosocial  dwarfism  in  childhood  are  mani¬ 
festations  of  deprivation  of  food  or  nurturance  (Harris, 
1982).  Victims  of  maltreatment  also  suffer  from  psychologi- 
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cal  harm  and  will  be  prone  towards  violence  themselves. 
Studies  support  the  impression  that  victims  of  maltreatment 
will  subsequently  maltreat  their  own  children  ( Altemeier , 
O'Connor,  Vietze,  1982;  Council  on  Scientific  Affairs, 

1985;  Hunter  et  al.,  1978;  Oates  et  al.,  1979;  Widom, 
1989).  One  study  estimates  that  as  many  as  30%  of  child 
maltreatment  victims  will  become  perpetrators  as  parents 
(Egeland,  1988).  The  costs  of  child  maltreatment  are  enor¬ 
mous  and  far-reaching  and  include  both  human  costs  and  such 
costs  to  society  as  medical/psychological  treatment  for  vic¬ 
tims  and  perpetrators,  court  time,  use  of  the  penal  system, 
and  the  lost  productivity  of  directly  affected  members  of 
society.  In  1976,  the  cost  to  society  of  caring  for  chil¬ 
dren  brain  damaged  from  abuse  was  approximately  4.2  billion 
dollars  annually  (Rosenberg,  Meyers,  Shackleton,  1982). 

Once  a  child  is  identified  as  having  been  maltreated, 
it  is  important  that  an  intervention  take  place  to  protect 
the  child  and  that  the  whole  family  receive  treatment. 
Without  such  precautions,  50%  of  physically  abused  children 
will  experience  more  maltreatment  and  10%  will  die  from  it, 
a  death  rate  100  times  higher  than  the  overall  death  rate 
from  maltreatment  estimated  by  NIS-2  (Rosenberg,  Meyers, 
Shackleton,  1982;  U.S.  Dept.  HHS,  1988).  Ideally,  treat¬ 
ment  in  child  maltreatment  cases  involves  counseling  for  the 
entire  family,  not  just  the  perpetrator.  In  order  to  iden¬ 
tify  cases  and  implement  proper  protection  and  treatment,  a 
team  effort  by  pediatricians,  nurses,  child  psychiatrists. 
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psychologists,  social  workers,  teachers,  attorneys,  and 
child  care  workers  is  necessary  (U.S.  Dept.  HHS,  1981). 

Given  proper  intervention  and  therapy,  it  is  estimated  that 
as  many  as  "90%  of  child  abuse  and  neglect  cases  respond" 
(Robert  Wood  Johnson  Foundation,  1977). 

Even  with  successful  therapy  that  prevents  repeat  epi¬ 
sodes,  once  a  child  is  maltreated  he  or  she  is  likely  to 
carry  deep  physical  and/or  emotional  scars.  The  only  way  to 
prevent  this  from  happening  is  to  prevent  the  first  episode 
of  child  maltreatment  itself  (Altemeier  et  al.,  1984). 
Dubowitz  wrote, 

Both  financial  and  human  costs  associated  with 
child  maltreatment,  although  crudely  estimated, 
are  staggering.  Prevention  is,  therefore,  attrac¬ 
tive  as  a  way  of  reducing  these  costs  of  child 
maltreatment.  In  addition,  there  is  the  possibil¬ 
ity  that  early  efforts  to  enhance  family  function¬ 
ing  could  be  more  effective  than  interventions 
after  maltreatment  has  already  occurred  (Dubowitz, 

1989) . 

Altemeier  et  al.  agreed  that  prevention  is  better  than 
treatment . 

Actually,  preventing  abuse  before  it  starts  may  be 
easier  than  stopping  it.  Many  of  the  factors 
which  apparently  predispose  to  parenting  disorders 
are  likely  to  be  increased  because  of  the  mal¬ 
treatment  (Altemeier  et  al.,  1979). 

The  Select  Panel  for  the  Promotion  of  Child  Health  concurred 

and  advocated  prevention  as  one  of  their  goals. 

Because  therapy  of  this  kind  is  expensive,  espe¬ 
cially  if  it  involves  residential  treatment,  most 
health  and  social  service  experts  stress  the  need 
for  better  preventive  programs,  based  on  early 
assessment  of  family  risk,  home  health  visiting  by 
public  health  nurses,  social  workers  or  lay  visi- 
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tors,  and  vigorous  community-based  campaigns  of 
education  and  crisis  management  (U.S.  Dept.  HHS, 

1981)  . 

Prevention  programs  fall  into  three  broad  categories, 
primary,  secondary,  and  tertiary.  Primary  prevention  tar¬ 
gets  the  population  at  large  and  takes  the  form  of  community 
service  announcements  about  resources  for  parents,  tapes  and 
demonstrations  on  good  parenting,  and  efforts  to  heighten 
public  awareness  about  the  dangers  of  maltreating  children. 
Secondary  prevention  programs  target  people  deemed  to  be  at 
high  risk  of  maltreating  their  children.  These  programs 
strive  to  improve  parenting  skills  and  family  functioning. 
Examples  of  these  include  latch-key  children  programs,  pro¬ 
grams  for  pregnant  teenagers,  and  crisis  intervention  serv¬ 
ices  like  hot-lines.  Tertiary  prevention  involves  rehabili¬ 
tation  of  those  known  to  have  maltreated  their  children 
along  with  psychological  treatment  of  the  victims  in  order 
to  foster  their  growth  and  decrease  the  likelihood  that 
they,  in  turn,  will  harm  their  own  future  children.  Child 
protective  services,  foster  care,  and  legal  prosecution  fall 
into  the  realm  of  tertiary  prevention  (Dubowitz,  1989). 

Of  the  three  types  of  prevention  programs,  primary  pre¬ 
vention  is  used  the  least  in  the  United  States.  These  types 
of  programs  have  not  been  carefully  evaluated  but  it  is  ex¬ 
pensive  and  difficult  to  effectively  reach  an  entire  popula¬ 
tion  (Dubowitz,  1989).  Most  programs  fall  into  the  category 
of  secondary  prevention.  These  programs  try  to  identify 
those  families  in  their  population  that  are  at  high  risk  and 
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then  provide  these  families  with  support  services.  These 
programs  report  a  decrease  in  child  abuse  and  neglect  (Dubo- 
witz,  1989).  Heifer  concluded  after  a  review  of  such  pro¬ 
grams,  however,  that  thorough  evaluation  research  into  these 
programs  are  still  needed  in  order  to  judge  their  effective¬ 
ness  (Heifer,  1982).  Tertiary  programs  are  also  common  in 
the  United  States.  They  encompass  actions  to  punish  the 
perpetrator,  protect  the  victim,  and  provide  therapy  for 
both,  but  these  programs  need  thorough  evaluations  as  well 
(Dubowitz,  1989). 

Evidence  does  exist,  though,  that  supports  secondary 
prevention  as  an  effective  means  of  deterring  maltreatment. 

A  prospective  study  in  New  Zealand  by  Monaghan,  Gilmore, 
Muir,  et  al.  concluded  that  interventions  decreased  the  rate 
of  maltreatment.  The  Stage  I  group  consisted  of  200  fami¬ 
lies,  none  of  whom  received  any  form  of  intervention.  At 
follow-up  two  years  later,  52%  of  those  families  judged  to 
be  High  Risk  at  the  start  of  the  study  had  experienced  an 
adverse  outcome  characterized  by  removal  of  a  child  from  the 
home  for  more  than  6  months.  Nine  percent  of  those  charac¬ 
terized  as  being  at  Moderate  Risk  experienced  an  adverse 
outcome  while  5%  of  those  at  Low  Risk  and  0%  of  those  at  No 
Risk  experienced  the  same  outcome.  The  Stage  II  group  con¬ 
sisted  of  300  families,  all  of  whom  received  such  interven¬ 
tions  as  a  support  group  lead  by  a  social  worker,  access  to 
day  care  facilities,  support  from  volunteers,  and  "consulta¬ 
tive  resources."  In  this  intervention  population,  20%  of 
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the  High  Risk  group  and  2%  of  the  Moderate  Risk  group  expe¬ 
rienced  an  adverse  outcome  while  none  of  the  Low  or  No  Risk 
groups  had  such  an  experience.  The  difference  in  the  rates 
of  adverse  outcome  between  the  intervention  and  non¬ 
intervention  groups  was  significant  at  p<0.01  (Monaghan, 
Gilmore,  Muir,  et  al.,  1986). 

The  best  study  done  to  date  in  the  United  States  on 
secondary  prevention  was  done  by  Olds  and  colleagues  in  the 
Appalachian  region  of  New  York  state.  Four  randomized 
treatment  groups,  each  consisting  of  from  90  to  116  first¬ 
time  mothers,  were  studied.  Group  1  was  a  no-treatment  con¬ 
trol  group.  Group  2  received  free  transportation  to  regular 
prenatal  and  well-baby  visits.  Group  3  received  the  free 
transportation  along  with  regular  home  visits  by  a  nurse 
during  the  pregnancy.  The  fourth  group  received  the  same 
interventions  as  Group  3  but  also  received  nurse  home  visi¬ 
tation  during  the  first  two  years  of  the  newborns'  lives. 

In  all  four  groups,  the  mothers  were  interviewed  at  regis¬ 
tration  to  gather  demographic  and  background  information; 
the  children  were  weighed  and  measured  at  6 ,  12,  and  24 
months;  the  children  were  screened  by  an  infant  specialist 
at  1  and  2  years  of  age  for  developmental  and/or  sensory 
problems;  and  the  state  child  abuse  registries  were  searched 
for  any  reports  of  maltreatment  on  these  children.  In  this 
study,  19%  of  the  High  Risk  mothers  (poor,  unmarried  teens) 
in  the  comparison  group  (Groups  1  and  2)  were  reported  for 
abusing  their  babies  in  the  first  two  years  of  life  compared 
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to  4%  of  the  High  Risk  group  who  received  prenatal  or  prena- 
tal/postpartum  nurse  visitation  (p=0.07).  Among  all  of  the 
women  in  the  study,  those  in  Treatments  3  and  4  showed  more 
concern  for  their  babies'  problems  (p=0.05)  and  reported 
that  their  babies  had  better  dispositions  (p=0.04),  while 
their  babies  were  brought  into  the  emergency  room  less  often 
( p=0 . 04 )  and  had  fewer  accidents  and  poisonings  (p=0.03) 
(Olds,  Henderson,  Chamberlin,  et  al.,  1986a). 

Programs  for  the  population  at  large,  primary  preven¬ 
tion,  are  not  only  expensive  and  difficult  to  implement,  but 
they  tend  to  be  more  superficial.  The  intensive,  expensive 
secondary  prevention  programs  are  more  likely  to  have  a 
positive  impact  on  families.  However,  it  would  be  an  inef¬ 
ficient  use  of  resources  to  try  to  implement  them  on  a  glo¬ 
bal  basis  as  primary  prevention  since  most  families  do  not 
need  them.  Before  secondary  prevention  can  be  used  effec¬ 
tively,  high  risk  populations  who  will  need  the  interven¬ 
tions  must  be  identified.  Altemeier  and  colleagues  have 
summarized  prevention  as  a  three  step  process: 

1)  identification  of  risk  factors. 

2)  identification  of  high  risk  families  with  the  risk  fac¬ 
tors  . 

3)  correction  of  deviant  elements  in  the  families  that  are 
at  the  root  of  the  risk  factors  (Altemeier  et  al., 

1979) . 

The  question  next  arises  as  to  when  the  optimal  time  is 
to  screen  families  and  intervene  on  behalf  of  the  high  risk 
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ones.  Heifer  has  identified  three  times  in  the  lives  of 
parents  when  they  are  most  accessible  for  mass  screening: 
during  schooling,  during  prenatal  care  or  delivery  of  a 
child,  and  when  a  child  is  first  entering  the  school  system 
(Heifer,  1976b).  Since  43%  of  maltreated  children  are  5  or 
younger,  to  wait  until  a  child  enters  the  school  system 
means  failing  to  prevent  almost  half  of  the  cases.  To  in¬ 
tervene  during  a  parent's  schooling  means  to  intervene  on 
parents  who  have  children  at  various  ages,  including  school- 
age  and  pre-conception.  To  try  to  help  people  who  have  yet 
to  start  their  families  is  of  questionable  effectiveness. 

The  ideal  time  to  screen  families  then  is  during  the  perina¬ 
tal  period  (Lynch,  Roberts,  1982).  Concluded  Gray  and  co¬ 
workers  , 

"Perinatal  assessment  and  early  consistent  inter¬ 
vention  with  families  identified  as  high  risk  for 
abnormal  parenting  practices  significantly  im¬ 
proves  the  infants'  chances  of  escaping  serious 
physical  injury  (Gray  et  al.,  1976)." 

Prior  Attempts  to  Predict  Maltreatment 

The  success  of  prevention  programs  for  a  large  part 
depends  on  proper  screening  and  accurate  identification  of 
those  families  who  are  at  high  risk.  Prior  research  indi¬ 
cate  that  the  development  of  such  an  instrument  is  possible. 
One  study  of  families  who  presented  with  young  children  to  a 
pediatric  emergency  room  found  that  "a  simple,  brief,  objec¬ 
tive  assessment  may  be  made  in  the  emergency  room  setting  to 
determine  which  patients  are  at  increased  risk  for  being 
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abused  in  the  future  (Rosenberg,  Meyers,  Shackleton,  1982)." 
Since  an  abusing  parent  has  more  incentive  than  a  non¬ 
abusing  parent  to  be  less  than  truthful  when  questioned, 
data  obtained  from  families  before  any  adverse  event  occurs 
may  be  more  valid  than  those  obtained  after  maltreatment  has 
begun,  and  prediction  of  those  at  risk  might  actually  be 
more  accurate  than  identification  of  those  parents  already 
maltreating  their  children  but  who  as  yet  have  not  been 
identified  (Altemeier  et  al.,  1979). 

With  a  general  consensus  that  identification  of  those 
at  high  risk  is  possible,  several  attempts  have  been  made  in 
the  past  to  develop  such  an  instrument.  Heifer  outlined 
three  typical  methods:  the  self-administered  questionnaire, 
the  standardized  interview,  and  observational  checklists 
(Heifer,  1987).  In  reviewing  past  research  on  prediction, 
Leventhal  found  that  the  eleven  studies  he  examined  fell 
into  one  of  four  categories:  a  checklist  of  socioeconomic 
factors,  a  structured  interview,  unstructured  clinical  judg¬ 
ment  and  structured  clinical  judgment  (Leventhal,  1988). 
These  two  classification  schemes  can  be  combined  as  follows: 
socioeconomic  evaluation,  the  self-administered  question¬ 
naire,  the  semi-structured  interview,  unstructured  clinical 
judgment,  and  structured  clinical  judgment. 

In  this  combined  classification  scheme,  the  five  cate¬ 
gories  are  defined  by  two  features:  1)  by  how  the  data  are 
gathered  (since  this  element  can  greatly  limit  the  useful¬ 
ness  of  an  instrument)  and  2)  by  how  judgments  of  level  of 
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risk  are  made. 

Socioeconomic  evaluations  are  those  screening  instru¬ 
ments  whose  contents  are  based  solely  on  social  and  economic 
factors  without  regard  to  past  experiences,  psychological 
make-up,  or  attitudes.  The  data  for  these  instruments  are 
usually  gathered  in  a  questionnaire  or  as  part  of  routine 
medical  care.  The  judgments  about  level  of  risk  are  based 
on  a  specific  set  of  criteria  that  define  High  Risk  families 
as  those  with  poor  social  supports  and/or  a  lower  economic 
class . 

The  self-administered  questionnaire  is  a  list  of  ques¬ 
tions  that  parents  are  given  and  allowed  to  answer  by  them¬ 
selves.  Also  included  in  this  category  are  interviews  where 
the  interviewer  asks  a  set  list  of  questions  and  records  the 
responses  verbatim  with  no  value  judgments  made  about  a  pa¬ 
tient's  veracity  and  no  prompting  for  responses;  this  type 
of  structured  interview  could  also  be  called  an  oral  ques¬ 
tionnaire  and  could  be  used  for  illiterate  parents. 

A  semi-structured  interview  is  defined  as  an  evaluation 
that  requires  an  evaluator  (usually  a  social  worker,  physi¬ 
cian,  or  nurse  trained  in  evaluating  high  risk  families)  to 
conduct  a  special  interview  with  the  parents.  This  inter¬ 
view  is  not  part  of  the  family's  routine  medical  care  but  is 
conducted  specifically  to  assess  the  risk  of  child  maltreat¬ 
ment.  After  the  interview,  which  is  open-ended  as  opposed 
to  a  questionnaire,  the  evaluator  has  a  specific  set  of 
criteria  to  follow  in  assigning  a  level  of  risk. 
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The  data  for  both  structured  and  unstructured  clinical 
judgments  are  gathered  during  a  clinician's  routine  care  of 
the  patient.  Thus,  the  evaluator  must  be  a  clinician  (e.g. 
nurse,  pediatrician,  obstetrician/gynecologist,  or  social 
worker)  caring  for  a  member  of  the  family;  this  clinician 
need  not  be  specially  trained  to  evaluate  families  for  risk 
of  maltreatment.  No  specific  "child  maltreatment"  interview 
is  conducted.  Rather,  all  data  are  obtained  during  the 
course  of  a  continuing  medical  relationship.  In  unstruc¬ 
tured  clinical  judgment,  the  overall  level  of  risk  is  based 
on  the  clinician's  experience  without  specific  guidelines  to 
follow.  In  structured  clinical  judgment,  the  clinician  must 
assess  the  family  on  specific  items  and  follow  established 
guidelines  in  scoring  the  items  in  order  to  determine  a  lev¬ 
el  of  risk. 

Socioeconomic  evaluation 

Although  some  research  has  discounted  the  value  of  bas¬ 
ing  a  family's  risk  status  solely  on  socioeconomic  factors 
(Steinberg,  Catalano,  Dooley,  1981),  Garbarino  and  Sherman 
did  just  that  in  proposing  to  identify  families  at  high  risk 
of  maltreatment  by  the  neighborhood  in  which  they  live. 

With  the  premise  that  child  maltreatment  is  a  social  prob¬ 
lem,  whole  neighborhoods  (and  the  residing  families)  are 
classified  as  "'low  risk'  if  they  help  support  families,  and 
'high  risk'  if  they  work  against  families"  (Table  1)  (Gar¬ 
barino,  Sherman,  1980).  Incidents  of  child  abuse  and  ne¬ 
glect  were  plotted  on  a  map  to  identify  neighborhoods  with  a 
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high  density  of  maltreatment  events.  Their  research  con¬ 
cluded  that  compared  to  Low  Risk  neighborhoods,  High  Risk 
neighborhoods  were  less  tranquil,  had  lower  rent,  had  no 
screens  on  the  windows  of  homes,  had  a  dichotomy  between 
home  ownership  and  home  rental ,  were  perceived  by  the  resi¬ 
dents  to  be  filled  with  more  unfriendly  neighbors,  had  a 
higher  involvement  of  families  but  a  weaker  family  unit,  and 
were  experiencing  a  lot  more  change  and  deterioration  (Gar- 
barino,  Sherman,  1980). 

This  form  of  screening  is  clearly  not  applicable  to 
screening  an  inner  city  population  since  most  families  would 
be  labeled  as  High  Risk,  although  statistically  and  clini¬ 
cally,  most  of  the  families  would  not  experience  maltreat¬ 
ment.  Furthermore,  this  technique  categorizes  people  not 
individually  but  in  large  groups,  ignoring  individual  varia¬ 
tion  and  mitigating  factors.  This  method  also  will  misclas- 
sify  as  Low  Risk  every  middle-  or  upper-class  family  at  risk 
of  maltreatment  despite  any  existing  family  pathology. 

While  this  study  may  be  interesting  as  descriptive  research, 
its  utility  as  a  screening  technique  is  minimal  at  best. 

The  self -administered  questionnaire 

The  second  method  of  assessing  parents  for  risk  of  mal¬ 
treatment  is  to  ask  the  parents  (usually  the  mother)  to  com¬ 
plete  a  questionnaire.  This  self-administered  questionnaire 
technique  has  been  employed  several  times  by  various  re¬ 
searchers.  Schneider,  Hoffmeister,  and  Heifer  used  a  74- 
item  questionnaire  in  the  peripartum  period.  The  questions 
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clustered  into  the  six  categories  of  self-esteem,  social 
isolation,  childhood  experience,  depression/crisis,  expecta¬ 
tions  of  children,  and  expectations  of  parenthood  (Schneid¬ 
er,  Hoffmeister,  Heifer,  1976).  Dean  and  others  at  3  to  4 
months  postpartum  used  both  a  questionnaire  that  examined 
maternal  attitudes  along  with  an  in-home  visit  by  a  social 
worker  who  gauged  mother-child  interactions  (Dean,  MacQueen, 
Mitchell  et  al . ,  1978).  Gabinet  administered  the  Minnesota 
Multiphasic  Personality  Inventory  to  try  to  outline  a  per¬ 
sonality  profile  of  the  child  abuser.  In  a  comparison  of 
High  Risk  parents  (chosen  by  unstructured  clinical  judg¬ 
ment),  known  child  abusers,  and  psychiatric  outpatients  with 
no  past  history  of  abusing,  there  was  a  striking  similarity 
between  the  three  groups  leading  the  researcher  to  conclude, 
"There  is  no  one  abusive  personality ...( Child  abuse  is)  pre¬ 
dictable  more  from  history  and  other  samples  of  behavior 
rather  than  by  personality  testing  (Gabinet,  1979)."  The 
160-item  Child  Abuse  Potential  Inventory  developed  by  Mil¬ 
ner,  Gold,  and  Wimberly  is  designed  not  as  a  perinatal 
screening  instrument  of  those  at  risk,  but  as  a  tool  to 
identify  those  parents  who  have  already  abused  their  chil¬ 
dren  (Milner,  Gold,  Wimberly,  1986). 

Aside  from  the  individual  shortcomings  discussed  above, 
questionnaires  have  some  intrinsic  difficulties.  Like  in¬ 
terviews,  they  require  the  parent(s)  to  be  cooperative  and 
actively  participating  in  the  screening.  Those  with  abusive 
tendencies,  however,  may  be  the  ones  least  willing  to  be 
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screened.  Questionnaires  also  offer  the  easiest  opportunity 
for  parents  to  lie;  with  a  questionnaire,  people  have  the 
time  to  carefully  study  the  responses  and  mark  the  ones  that 
they  feel  reflect  good  parents  rather  than  those  responses 
that  reflect  themselves.  In  addition,  questionnaires  cannot 
detect  behaviors  indicative  of  untruthfulness  or  violence 
which  a  one-to-one  interaction  between  clinician  and  parent 
may  detect.  The  value  of  a  questionnaire  is  completely  at 
the  mercy  of  the  willingness  of  the  parent  to  reveal  herself 
(or  himself)  on  paper. 

The  semi -structured  interview 

Many  different  forms  of  the  semi-structured  interview 
have  been  developed.  An  early  one  was  devised  by  Monaghan 
and  Couper-Smartt  in  New  Zealand  and  involved  an  in-depth 
interview  of  mothers  conducted  during  the  eight-day  postpar¬ 
tum  hospital  stay.  The  interview  was  conducted  by  a  pedia¬ 
trician,  a  social  worker,  and  a  family  psychiatrist,  members 
of  a  Child  Care  Unit  who  specifically  worked  with  women  hav¬ 
ing  trouble  with  parenting.  The  patients  interviewed  were 
those  referred  by  their  obstetrician  after  exhibiting  some 
form  of  parenting  distress,  a  subjective  judgment  on  the 
obstetrician's  part.  The  evaluation  by  the  Child  Care  Unit 
included  asking  about  the  mother's  own  childhood,  her  social 
supports,  and  her  expectations;  a  parent  was  judged  to  be 
at  high  risk  of  maltreating  a  child  if  she  had  two  or  more 
of  the  ten  examined  risk  factors  (Monaghan  and  Couper-Sma¬ 
rtt,  1977). 


25 


While  the  researchers  report  that  their  criteria  are 
"predictive,"  their  method  does  not  have  widespread  applica¬ 
tion.  First,  no  maternity  ward  in  the  United  States  has  the 
luxury  of  an  eight-day  postpartum  observation  period  during 
which  parenting  and  bonding  can  be  gauged.  At  Yale-New 
Haven  Hospital,  the  average  postpartum  stay  is  two  days  for 
vaginal  births  and  five  days  for  cesarean  births.  Second, 
the  application  of  this  interview  required  three  highly 
trained  family  experts,  expertise  which  all  programs  are  not 
likely  to  have  available  to  them.  Third,  this  method  re¬ 
quires  that  the  patient  be  cooperative  and  willing  to  par¬ 
ticipate  in  an  in-depth  interview.  Finally,  Monaghan  and 
Couper-Smartt ' s  interview  is  not  really  a  screening  instru¬ 
ment.  Their  interview  was  not  given  to  all  parents  (perhaps 
because  of  some  of  the  drawbacks  discussed  above)  but  rather 
just  to  those  already  judged  to  be  somehow  troubled  in  their 
parenting.  This  judgment  of  troubled  parenting  in  need  of 
follow-up  was  done  by  obstetricians  in  an  unstructured  way 
and  is  really  the  initial  screening  step. 

This  approach  to  the  interview  technique  was  later 
modified  by  Monaghan,  Gilmore,  Muir,  et  al.  They  had  a  team 
of  experts  interview  pregnant  women  and  then  complete  a 
nine-item  screening  questionnaire  afterwards  (Table  1).  In 
testing  the  instrument  over  a  two  year  follow-up  period,  a 
negative  outcome  was  measured  as  relinquishment  of  custody 
of  the  child,  referral  to  the  Department  of  Social  Welfare, 
referral  to  Child  Protection,  or  the  involvement  of  family 
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court  or  a  social  worker.  This  study  found  a  positive  pre¬ 
dictive  accuracy  of  90%  for  those  labeled  High  Risk  and  67% 
for  those  labeled  Moderate  Risk,  and  a  negative  predictive 
accuracy  of  36%  for  those  labeled  Low  Risk  and  88%  for  those 
labeled  No  Risk  (Monaghan,  Gilmore,  Muir,  et  al . ,  1986). 

Monaghan  et  al.'s  instrument  suffers  from  the  same 
problem  of  needing  a  team  of  experts  to  evaluate  the  patient 
and  a  patient  willing  to  be  interviewed.  There  also  was 
poor  discrimination  between  the  Moderate  and  Low  risk  cate¬ 
gories;  67%  of  the  former  and  64%  of  the  latter  went  on  to 
have  a  negative  outcome.  These  rates  may  be  even  higher  if, 
as  often  happens,  not  all  cases  of  maltreatment  were  report¬ 
ed  to  agencies.  Finally,  this  instrument  was  developed  us¬ 
ing  a  population  in  New  Zealand  that  is  "economically  advan¬ 
taged  and  has  well-developed  health  and  social  services" 
(Monaghan,  Gilmore,  Muir,  et  al . ,  1986).  While  the  screen¬ 
ing  instrument  may  identify  those  at  high  risk  of  parenting 
difficulty  in  this  middle-  to  upper-class  population  with 
socialized  medicine,  it  is  not  safe  to  extrapolate  the  con¬ 
clusions  to  an  inner  city  population  in  the  U.S.  that  has 
limited  access  to  health  care  and  social  services.  Identi¬ 
fiable  risk  factors  in  one  population  may  not  be  the  same  in 
the  second  population. 

Altemeier  et  al.  developed  a  45-minute  interview  de¬ 
signed  to  be  given  to  mothers  in  the  prenatal  period.  This 
interview  examined  eight  areas  including  the  mother's  own 
nurture  as  a  child,  her  feelings  about  the  pregnancy,  and 
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any  substance  abuse  problems  (Altemeier,  Vietze,  Sherrod,  et 
al.,  1979).  This  format  was  later  revised  to  a  35  minute 
interview  with  the  data  clustering  into  six  predictive 
areas:  subjective  impression  of  the  interviewer  (most  pre¬ 

dictive),  residency  transience,  untruthfulness,  disturbed 
childhood  nurturance,  unwanted  pregnancy,  and  increased  par¬ 
ent-child  exposure  (Table  1).  When  tested  on  1400  pregnant 
women,  Altemeier  and  colleagues  found  that  they  had  correct¬ 
ly  predicted  53%  of  the  abused  children  with  a  94%  false 
positive  rate  and  with  prediction  good  for  up  to  24  months 
(Altemeier,  O'Connor,  Vietze,  et  al.,  1984).  This  method 
uses  a  long  interview  that  is  not  practical  for  mass  screen¬ 
ing;  it  was  developed  more  with  research  than  screening  in 
mind.  The  results  also  yielded  an  unacceptably  high  false 
positive  rate.  It  is  of  note,  though,  that  the  most  predic¬ 
tive  category  was  the  unstructured  judgment  of  the  inter¬ 
viewer,  raising  the  possibility  of  using  that  alone  as  the 
screening  tool. 

At  the  University  of  Colorado,  Murphy,  Orkow  and  Nicola 
tried  screening  pregnant  women  using  the  Family  Stress 
Checklist  originally  developed  by  Schmitt  and  Carroll  (Table 
1).  This  interview  was  administered  by  an  experienced  so¬ 
cial  worker  and  had  a  positive  predictive  accuracy  of  52.6% 
and  a  negative  predictive  accuracy  of  96.6%  (Murphy,  Orkow, 
Nicola,  1985).  The  true  merits  of  this  checklist  are  hard 
to  discern,  however.  Those  judged  to  be  High  Risk  received 
interventions  such  as  parenting  classes  during  the  follow-up 
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period  (Orkow,  1985).  Thus,  it  is  possible  that  if  no  in¬ 
tervention  had  been  implemented,  the  positive  predictive 
accuracy  may  have  been  higher.  On  the  other  hand,  the  defi¬ 
nition  of  child  abuse  and  neglect  used  in  the  follow-up  in¬ 
cluded  such  criteria  as  cradle  cap  and  diarrhea  which  may 
have  incorrectly  categorized  some  well-cared  for  children  as 
maltreated  (Murphy,  Orkow,  Nicola).  A  final  pitfall  of  this 
instrument  is  that,  again,  it  requires  a  trained  social 
worker  to  administer  it. 

Avison,  Turner,  and  Noh  developed  a  20-question  screen¬ 
ing  interview  for  mothers  that  looked  for  "parental  maladap- 
tation"  (Table  1)  (Avison,  Turner,  Noh,  1986).  The  ques¬ 
tions  looked  at  social  supports  and  parenting  attitudes. 

When  tested  on  a  group  of  87  known  maltreaters  (maladaptive 
mothers)  and  100  controls  (presumed  to  be  well  adapted),  the 
interview  correctly  identified  96%  of  the  maladapted  mothers 
and  90%  of  the  comparison  mothers  for  an  overall  accuracy  of 
93%.  However,  a  test  of  the  predictive  validity  of  this 
instrument  was  not  conducted  for  "severe  ethical  and  practi¬ 
cal  difficulties"  and  the  authors  go  on  to  warn  that  this 
instrument  "cannot  be  regarded  as  a  clinical  or  diagnostic 
instrument  or  used  for  such  purposes"  (Avison,  Turner,  Noh, 
1986) . 

Unstructured  clinical  judgment 

Unstructured  clinical  judgment  goes  on  daily  when  phy¬ 
sicians  and  nurses  refer  families  to  any  type  of  social  work 
evaluation  or  intervention  because  of  their  own  sense,  based 
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on  experience,  that  a  family  is  in  need  of  help.  Leventhal, 
Garber,  and  Brady  studied  such  judgments  at  Yale-New  Haven 
Hospital  by  doing  a  retrospective,  longitudinal  cohort  study 
of  babies  referred  at  birth  (based  on  unstructured  clinical 
judgments)  by  pediatricians,  nurses,  and  social  workers  to 
the  hospital's  child  abuse  committee,  known  as  the  DART 
(Detection,  Assessment,  Reporting,  Treatment)  Committee. 
Compared  to  the  matched  control  group,  by  the  fourth  birth¬ 
day  more  of  the  referred  children  experienced  actual  child 
maltreatment  than  the  control  group  (23%  vs.  8%  with  a 
matched  odds  ratio  of  3.1)  (Leventhal,  Garber,  Brady,  1989). 
While  unstructured  clinical  judgment  is  the  most  common 
method  of  screening,  and  is  actually  a  good  method,  it  re¬ 
lies  heavily  on  a  clinician's  experience  and  intuition. 
Younger,  less  experienced  clinicians  are  less  likely  to  do 
as  well  in  detecting  which  families  are  at  risk. 

Structured  clinical  judgment 

To  capitalize  and  improve  on  the  effectiveness  of  un¬ 
structured  judgments,  several  researchers  have  tried  to 
create  instruments  that  structure  clinical  judgment.  This 
approach  benefits  from  the  clinician's  personal  interactions 
with  the  family  yet  guides  the  clinician  as  to  which  areas 
of  family  life  to  pay  particular  attention.  Rosenberg  et 
al.  used  this  approach  in  the  emergency  room  to  evaluate  all 
children  younger  than  24  months  who  came  in  and  their  fami¬ 
lies.  An  8-item  evaluation  to  be  used  by  nurses  examined 
the  state  of  the  child's  care,  the  parents'  behaviors,  and 
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the  quality  of  parent-child  interactions  with  a  42%  rate  of 
later  maltreatment  in  the  High  Risk  group  (Rosenberg,  Mey¬ 
ers,  Shackleton,  1982).  Since  this  instrument  looks  for 
signs  of  past  maltreatment  (bruises,  poor  child  care)  it  is 
more  a  tool  for  tertiary  prevention,  rather  than  secondary 
prevention.  Nonetheless,  it  is  one  successful  application 
of  the  principles  of  structuring  clinical  judgment. 

Murphy  et  al.  and  Gray  et  al .  each  developed  predictive 
instruments  that  structure  clinical  information  in  the  post¬ 
partum  period.  The  instrument  by  Murphy  and  colleagues  is 
based  on  a  review  of  objective  information  coded  by  ward 
clerks  into  the  birth  records  of  all  children  and  is  com¬ 
posed  of  11  factors,  such  as  social  class,  age  of  the  moth¬ 
er,  and  birth  weight  of  the  infant,  found  by  the  researchers 
to  be  correlated  with  later  abuse  (Table  1)  (Murphy,  Jen¬ 
kins,  Newcombe,  et  al.,  1981).  While  this  instrument 
screens  families  in  the  ideal  postpartum  period,  it  does  not 
make  use  of  clinical  interactions  or  impressions  but  is 
wholly  reliant  on  objective  biographical  data,  thereby  not 
maximizing  the  full  potential  of  clinical  judgment.  Gray 
and  colleagues  began  to  develop  a  screening  instrument  that 
also  evaluated  families  in  the  postpartum  period  but  uti¬ 
lized  information  on  the  families'  interactions  with  hospi¬ 
tal  staff.  This  instrument  included  such  items  as  disap¬ 
pointment/pleasure  with  the  baby's  gender,  parental  reac¬ 
tions  to  infant  crying,  parental  in-hospital  care  of  the 
baby,  and  attempted  manipulation  of  the  staff  by  the  parents 
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(Gray,  Cutler,  Dean,  et  al.,  1976).  While  this  instrument 
appeared  very  promising,  it  was  never  fully  developed  or 
tested  (written  communication,  Aug.  7,  1989). 

Lealman,  Haigh,  Phillips,  et  al.  took  a  similar  ap¬ 
proach  to  Gray  and  devised  a  checklist  of  behaviors  and 
characteristics  of  parents  to  screen  for  risk  of  later  mal¬ 
treatment.  This  checklist  was  composed  of  10  items  avail¬ 
able  from  maternity  notes  and  was  applied  during  the  dis¬ 
charge  of  mothers  from  a  maternity  ward.  Risk  factors  in¬ 
cluded  maternal  age,  prenatal  care,  and  marital  status  (Ta¬ 
ble  1)  (Lealman,  Haigh,  Phillips,  et  al.,  1983).  This  in¬ 
strument  was  never  properly  tested,  either.  It  was  de¬ 
veloped  and  used  as  part  of  an  prevention  program;  thus, 
most  of  those  identified  as  High  Risk  received  some  sort  of 
intervention  during  the  follow-up  period,  making  it  impossi¬ 
ble  to  calculate  the  true  sensitivity,  specificity,  and 
positive  predictive  accuracy  of  the  scale. 

In  their  research  into  risk  factors  for  child  maltreat¬ 
ment,  Browne  and  Saqi  concluded,  "Our  findings  suggest  that 
family  stress  is  not  a  sufficient  explanation  for  child 
abuse  (Browne,  Saqi,  1988)."  They  constructed  a  12-item 
checklist  designed  to  be  used  by  nurses  after  interviewing 
the  mother  during  the  postpartum  period  to  identify  those  at 
high  risk  of  later  maltreating  their  newborn  baby.  This 
instrument  included  parental  age,  history  of  family  vio¬ 
lence,  and  financial  problems  in  the  family  and  was  designed 
with  the  population  of  Surrey,  England  in  mind  (Table  1) 
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(Browne,  Lowton,  1987).  A  study  of  the  predictive  validity 
of  the  scale  on  14,283  newborns  and  their  families  yielded  a 
sensitivity  of  81%,  a  specificity  of  94%  but  a  positive  pre¬ 
dictive  accuracy  of  only  7%  (Browne,  Saqi ,  1988).  The  poor 
predictive  accuracy  of  the  instrument  may  have  been  due  in 
part  to  the  low  prevalence  of  child  maltreatment,  but  also 
may  have  resulted  from  the  inclusion  of  some  questionable 
risk  factors  in  the  scale  like  bottle  (rather  than  breast) 
feeding,  prematurity  (Leventhal,  Egerter,  Murphy,  1984),  and 
socioeconomic  status  (Steinberg,  Catalano,  Dooley,  1981; 
Egeland,  1979;  Daro,  1988)  and  the  exclusion  of  risk  factors 
felt  to  be  strong  predictors,  like  psychiatric  stress  (Krug- 
man,  Lenherr,  Betz,  et  al.,  1986)  and  whether  the  pregnancy 
was  planned  (Lealman,  Haigh,  Phillips,  et  al.,  1983;  Lynch, 
1976).  This  instrument  represents  a  good  beginning  in  the 
structuring  of  clinical  judgment,  but  may  benefit  in  terms 
of  improved  positive  predictive  accuracy  if  the  items  were 
refined  and  expanded. 

Prediction  of  those  at  risk  of  future  maltreatment  is 
an  important  first  step  in  secondary  prevention.  However,  a 
good  scale  for  screening  purposes  has  yet  to  be  developed. 
Wrote  Rosenberg  et  al.; 

Most  methods  of  detecting  child  abuse  are  very 
time  consuming  and  retrospective  in  nature.  A 
good  screening  procedure  has  not  yet  been  devised 
that  will  enable  busy  pediatric  facilities  to  de¬ 
tect  children  at  special  risk  for  abuse  (Rosen¬ 
berg,  Meyers,  Shackleton,  1982). 
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The  ideal  time  to  screen  families  is  during  the  post¬ 
partum  period,  before  any  adverse  event  can  occur  and  before 
the  families  feel  they  have  anything  to  hide.  Lynch  and 
Roberts  retrospectively  reviewed  the  birth  records  of  abused 
and  non-abused  children  and  identified  five  risk  factors 
present  at  birth  with  five  times  as  much  frequency  in  chil¬ 
dren  later  abused  as  compared  to  those  not  abused  (Table  1). 
From  this  they  concluded  that  families  at  high  risk  can  be 
identified  with  the  data  readily  available  and  routinely 
collected  on  postpartum  hospital  wards  (Lynch,  Roberts, 
1978).  The  work  of  Leventhal ,  Garber,  and  Brady  also  sup¬ 
ports  the  idea  that  screening  in  the  postpartum  period  can 
be  done  successfully  (Leventhal,  Garber,  Brady,  1989). 

The  ideal  method  to  screen  families  is  through  the  use 
of  structured  clinical  judgment  based  on  the  routine  inter¬ 
actions  of  patients  and  clinicians.  As  discussed  above, 
evaluations  based  on  specially  designed  interviews  are  hin¬ 
dered  by  their  length,  the  difficulty  in  gaining  access  to 
all  parents,  the  difficulty  in  gaining  the  participation  of 
all  parents  (especially  those  at  high  risk),  the  necessity 
of  training  many  experts  to  conduct  the  interviews,  and  the 
limitations  on  screening  every  parent  based  on  cost  and 
available  labor.  Self-administered  questionnaires  also  re¬ 
quire  active  parental  participation  that  may  be  hard  to  ob¬ 
tain  and  more  readily  lend  themselves  to  deception  on  the 
part  of  the  parents.  The  validity  of  socioeconomic  scales 
is  in  dispute;  inherently  they  cannot  screen  well  a  low- 
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income  inner-city  population.  Finally,  unstructured  clini¬ 
cal  judgments  have  been  shown  to  be  the  most  common  and  con¬ 
venient  method  to  use,  yet  they  are  in  need  of  improvement 
so  that  less  experienced  clinicians  can  also  predict  accu¬ 
rately.  Thus,  a  screening  instrument  based  on  structured 
clinical  judgment  that  focused  the  clinician's  attention  on 
valid,  readily  decipherable  risk  factors  would  be  the  ideal 
instrument . 

Until  now,  a  screening  instrument  based  on  structured 
clinical  judgment  designed  for  use  in  the  postpartum  period 
had  yet  to  be  developed  and  tested. 

Clinimetrics  and  the  Development  of  Rating  Scales 

A  screening  instrument  for  risk  of  child  maltreatment 
based  on  structured  clinical  judgment  would  fall  into  the 
category  of  clinical  rating  scales.  "A  useful  clinical 
scale,"  point  out  Hutchinson  and  colleagues,  "must  fulfill 
the  basic  scientific  requirements  of  a  measurement  yielding 
results  that  are  reproducible  and  valid  (Hutchinson,  Boyd, 
Feinstein,  et  al.,  1979)."  Unlike  scales  for  laboratory 
measurements  and  other  "hard"  data,  the  measurement  of  so- 
called  "soft"  data  has  not  enjoyed  enough  scientific  scruti¬ 
ny  to  lead  to  the  establishment  of  rigorous  criteria  for 
their  construction  and  use.  Hutchinson  feels: 

(This  is  due  to  the)  belief  that  the  subjective 
information  required  to  assess  function  is  too 
unreliable  to  merit  serious  scientific  considera¬ 
tion....  No  general  standards  have  been  established 
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to  appraise  rating  scales  for  clinical  phenomena 

(Hutchinson,  Boyd,  Feinstein,  et  al.,  1979). 

Feinstein  has  made  major  strides  in  the  field  by  defin¬ 
ing  the  issues  and  suggesting  preliminary  guidelines.  He 
proposes  the  term  "clinimetrics"  for  "the  measurement  of 
clinical  phenomena,"  subdividing  the  activity  into  mensura¬ 
tion  (the  acquisition  and  labeling  of  data)  and  quantifica¬ 
tion  (clinical  epidemiology)  (Feinstein,  1987).  The  devel¬ 
opment  of  a  clinical  rating  scale  would  fall  under  the  men¬ 
suration  subcategory  of  clinimetrics.  Feinstein  points  out 
that  even  though  formal  standards  have  been  lacking,  clini¬ 
cal  observations  have  long  been  recorded,  described,  catego¬ 
rized,  and  rated  (Feinstein,  1987). 

Since  clinical  observations  are  more  complex  than 
"hard"  data  like  laboratory  values,  their  measurement  is 
more  complex,  the  method  used  often  depending  on  the  purpose 
of  the  measurement.  A  rating  scale  can  indicate  presence/ 
absence,  magnitude  such  as  none/mild/moderate/severe,  or 
more  complex  descriptions  such  as  tumor/node/metastasis. 

The  function  of  a  clinimetric  scale  has  been  divided  into 
four  general  types  by  Feinstein.  The  first  is  to  describe 
the  status  of  a  disease;  this  would  include  diagnostic 
criteria  and  ratings  of  clinical  conditions.  The  second 
function  is  to  measure  change  in  a  disease,  sign,  or  symp¬ 
tom.  The  third  function  is  to  describe  prognosis,  and  the 
fourth  function  is  to  describe  a  treatment  protocol  (Fein¬ 
stein  ,  1987 )  . 


After  clarifying  the  function  of  the  clinimetric  in¬ 
strument,  the  outline  for  the  development  of  a  clinimetric 
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scale  as  proposed  by  Feinstein  is  as  follows.  First,  possi¬ 
ble  variables  to  be  included  on  the  scale  are  selected; 
some  are  retained  and  some  are  excluded.  Each  component 
variable  retained  is  then  described  in  its  own  scale.  These 
component  variables  must  then  be  combined  to  produce  an  out¬ 
put  (a  score  for  the  entire  instrument)  with  its  own  output 
scale . 

There  are  two  basic  methods  to  selecting  the  component 
variables  for  a  clinimetric  scale.  One  is  to  examine  what 
is  to  be  measured  and  then  gather  the  intrinsic  and  extrin¬ 
sic  evidence  that  the  clinical  phenomenon  exists.  Intrinsic 
evidence  would  describe  the  phenomenon,  such  as  the  exist¬ 
ence  of  enlarged  nodes  or  a  symptom  like  shortness  of 
breath.  Extrinsic  evidence  is  a  result  of  the  phenomenon, 
such  as  the  ability  to  care  for  oneself.  The  second  method 
to  selecting  component  variables  is  to  review  past  research 
in  the  field  and  gather  those  variables  felt  to  be  signifi¬ 
cant  markers  for  the  phenomenon  under  study  (Feinstein, 

1987)  . 

In  clinimetrics ,  the  original  scales  for  the  individual 
variables  are  often  ordinal  (ranked  but  without  equal  magni¬ 
tude  between  adjacent  ranks)  rather  than  dimensional  (each 
rank  is  of  the  same  magnitude)  since  clinical  phenomena  are 
often  with  nondimensional  outcomes  (e.g.  the  difference  be¬ 
tween  severe  and  moderate  pain  may  not  be  the  same  as  the 
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difference  between  moderate  and  mild  pain).  When  ordinal 
scales  are  combined  to  form  the  outcome  scale,  the  result  is 
what  Feinstein  calls  quasi-dimensional ,  an  ordinal  scale 
with  unequal  ranks  yet  with  the  illusion  of  having  equal 
magnitude  between  ranks  (Feinstein,  1987). 

Guidelines  do  exist  for  the  use  of  ordinal  scales  for 
clinical  measurements.  First,  the  elements  of  the  scale 
must  be  clearly  defined  and  mutually  exclusive;  any  event 
to  be  measured  on  the  scale  must  fit  into  one  and  only  one 
rank  on  the  scale.  Second,  the  ranks  on  the  scale  must  ex¬ 
ist  in  a  hierarchical  order;  if  not,  the  scale  is  nominal, 
not  ordinal.  Third,  the  scale  must  be  constructed  in  a 
meaningful  way  so  that  a  change  in  rank  on  the  scale  re¬ 
flects  a  clinically  meaningful  difference.  Fourth,  the 
scale  must  be  symmetrical  so  that  improvement  and  deteriora¬ 
tion  can  both  be  measured.  Finally,  if  other  related  meas¬ 
ured  outcomes  exist,  the  scale  must  produce  a  result  conso¬ 
nant  with  the  other  outcomes.  In  addition,  it  is  important 
to  know  the  clinical  significance  of  differences  in  scores 
and  the  expected  variation  in  scores  (Mackenzie,  Charlson, 
1986)  . 

There  are  several  methods  to  combine  component  varia¬ 
bles  into  an  outcome  measure.  One  is  to  sum  the  variables. 
Although  the  easiest  method,  this  is  not  always  the  best. 

As  Browne  and  Saqi  point  out; 

Unfortunately,  most  screening  procedures  using  a 
checklist  format  add  the  number  of  risk  factors 
present  and  obtain  a  simple  summation  score  that. 
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in  effect,  treats  all  risk  factors  the  same.... 

This  is,  of  course,  illogical  and  limits  the  use¬ 
fulness  of  the  checklist  (Browne,  Saqi ,  1988). 

Another  method  is  to  use  Boolean  clusters;  outcome  catego¬ 
ries  are  made  up  of  different  combinations  of  each  of  the 
component  variables.  A  tandem  profile  that  lists  the  result 
of  each  component  variable  in  the  outcome  variable  (such  as 
in  the  TNM  staging  system)  is  another  possible  method.  Fi¬ 
nally,  a  hierarchical  system  can  be  used  where  each  compo¬ 
nent  variable  is  analyzed  individually  in  a  specified  order. 
If  any  variable  exceeds  a  specified  cut-off  then  an  extreme 
is  reached  such  that  the  remaining  variables  can  be  ignored. 
This  is  how  cancer  staging  works  where  metastases  are  evalu¬ 
ated  first,  then  nodes,  then  tissue  pathology  (Feinstein, 
1987)  . 

"Despite  the  general  scientific  prejudice  against  soft 
data,"  writes  Feinstein,  "clinicians  (have)  often  gone  ahead 
and  created  indexes  for  the  soft  phenomena  investigated  in 
their  research  (Feinstein,  1987)."  Classic  examples  include 
the  Apgar  score,  the  Glascow  Coma  Scale,  the  Trauma  Index, 
the  Yale  Observation  Scale,  and  the  Dubowitz  score  for  ges¬ 
tational  age. 

Apgar 's  scale  for  describing  the  condition  of  a  newborn 
was  an  early  clinimetric  scale.  Her  scale  observes  a  new¬ 
born  at  one  minute  and  five  minutes  of  life  and  rates  the 
baby  on  five  component  variables,  each  of  which  has  an  indi¬ 
vidual  three  point  ordinal  scale.  The  component  variables 
were  selected  from  Apgar' s  observations  of  intrinsic  evi- 
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dence  of  newborn  condition  and  were  those  she  felt  were  easy 
to  determine  and  which  she  "considered  useful  (Apgar, 

1953)."  The  components  are  combined  by  direct  summation. 
Although  Apgar  found  one  component  variable,  heart  rate, 
more  prognostic  of  how  a  newborn  will  fare,  she  chose  to 
keep  her  scale  simple  and  not  weight  the  variables  (Apgar, 
1953) . 

The  Glascow  Coma  Scale  is  another  instrument  used  to 
describe  a  patient's  condition,  in  this  case,  level  of  con¬ 
sciousness.  There  are  three  component  variables  each  of 
which  is  recorded  on  an  ordinal  scale.  However,  the  compo¬ 
nent  scales  have  different  maximums  reflecting  relative 
weighting  being  given  to  the  variables  (Teasdale,  Jennett, 
1974).  Although  the  "motor"  variable  measuring  movement  is 
alone  considered  the  best  indicator  of  level  of  conscious¬ 
ness  (Jagger,  Jane,  Rimel,  1983),  the  Glascow  Coma  Scale 
incorporates  two  other  variables  since  the  motor  variable 
cannot  always  be  measured  (e.g.  if  the  patient  is  in  trac¬ 
tion).  The  output  score  is  obtained  by  a  summation  of  the 
component  variables.  This  scale  is  an  excellent  example  of 
how  structured  clinical  judgment  can  produce  more  inter¬ 
rater  agreement  than  simple  unstructured  clinical  judgment 
(Teasdale,  Jennett,  1974).  By  offering  a  standard  way  to 
describe  a  patient's  condition,  the  scale  allows  doctors  to 
avoid  "ambiguities  and  misunderstandings  (that  result)  when 
groups  of  patients  treated  by  alternative  methods  are  com¬ 
pared,  or  reported  from  different  centres  (Teasdale,  Jen- 
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nett,  1974)." 

The  Trauma  Index  by  Kirkpatrick  and  Youmans  is  another 
screening  instrument  designed  to  describe  a  patient's  condi¬ 
tion.  The  variables  were  selected  after  a  review  of  those 
items  that  clinicians  have  in  the  past  cited  in  their  hospi¬ 
tal  notes  when  assessing  a  patient's  level  of  trauma.  Those 
component  variables  easy  to  assess  were  retained,  and  the 
composite  score  was  derived  from  the  summation  of  the  indi¬ 
vidual  variables  (Kirkpatrick,  Youmans,  1971).  This  is  the 
same  approach  taken  by  McCarthy  in  developing  the  Yale  Ob¬ 
servation  Scale  for  degree  of  illness  in  the  febrile  child. 
McCarthy  and  colleagues  initially  selected  their  component 
variables  from  those  which  experienced  pediatricians  stated 
they  used  in  evaluating  a  child  with  a  fever.  After  test¬ 
ing,  only  those  which  were  "independent  and  significant  pre¬ 
dictors"  of  serious  illness  based  on  multiple  regression 
analysis  were  retained  (McCarthy,  Sharpe,  Spiesel ,  et  al., 
1982).  The  output  variable  is  again  a  summation  of  the  com¬ 
ponent  variables.  This  method,  concluded  McCarthy  and  col¬ 
leagues,  "can  be  used  to  study  clinical  judgment  in  other 
areas  of  pediatrics  (McCarthy,  Sharpe,  Spiesel,  et  al., 

1981)  . " 

An  interesting  rating  scale  is  the  Dubowitz  score  for 
gestational  age  as  modified  by  Sweet.  This  instrument  as¬ 
sesses  10  neurologic  features  and  11  physical  features  in 
newborns  less  than  24  hours  old.  The  summary  score  is  an 
estimate  of  the  newborn's  gestational  age  (Dubowitz,  Dubo- 
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witz,  Goldberg,  1970;  Sweet,  1979).  This  clinimetric  scale 
is  of  particular  interest  because  unlike  many  other  scales 
of  clinical  judgment,  there  are  "hard"  data  with  which  to 
compare  the  result.  When  compared  to  gestational  dating  by 
the  mother's  last  menstrual  period,  Dubowitz  found  a  corre¬ 
lation  coefficient  of  0.93  with  a  95  per  cent  confidence 
limit  of  +  2  weeks  (Dubowitz,  Dubowitz,  Goldberg,  1970). 

When  2  independent  evaluations  of  the  same  baby  were  com¬ 
pared,  the  95  per  cent  confidence  limit  of  the  average  score 
was  ±  1.4  weeks.  This  study  is  important  in  highlighting 
the  point  that  although  considered  "soft"  data,  clinical 
judgment  can  be  measured  in  a  reliable,  reproducible  fash¬ 
ion  . 


In  order  to  develop  a  screening  instrument  to  assess  a 
newborn's  risk  of  subsequent  maltreatment,  the  principles  of 
clinimetrics  discussed  above  must  be  followed.  The  function 
of  this  instrument  will  be  secondary  prevention,  to  screen 
the  population  to  identify  those  at  high  risk.  The  compo¬ 
nent  variables  will  be  selected  by  a  combination  of  select¬ 
ing  those  intrinsic  and  extrinsic  factors  felt  to  be  impor¬ 
tant  by  clinicians  and  by  a  critical  review  of  the  research 
literature  on  risk  factors  associated  with  child  maltreat¬ 
ment.  The  next  step  will  be  to  make  individual  scales  for 
each  component  variable  and  then  to  combine  the  variables 
into  an  output  score.  This  process  will  be  discussed  in  the 


next  section. 


CHAPTER  THREE 


METHODS 
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This  research  was  reviewed  and  approved  as  Protocol 
#5157  by  the  Human  Investigation  Committee  of  Yale  Universi¬ 
ty  School  of  Medicine  and  Yale-New  Haven  Hospital.  The  data 
were  analyzed  on  an  IBM  PS-2  using  the  SAS  statistical  pro¬ 
gram.  The  weighted  kappa  values  were  computed  using  the 
RATCATA  Computer  Program  for  Assessing  Rater  Agreement  and 
Bias  from  Contingency  Tables. 

The  development  and  testing  of  a  new  instrument  using 
structured  clinical  judgment  to  screen  for  risk  of  future 
maltreatment  was  conducted  in  4  phases.  Phase  I  was  the 
selection  of  the  component  variables  and  the  construction  of 
the  clinimetric  instrument.  Phase  II  was  the  piloting  of 
the  instrument  on  176  newborns  in  order  to  refine  the  varia¬ 
bles  and  individual  scales.  Phase  III  involved  the  use  of 
the  instrument  by  both  experts  and  non-experts  in  the  field 
of  child  maltreatment.  In  addition,  the  experts  gave  their 
overall  assessments  of  each  subject's  risk.  From  these 
data,  a  scoring  method  was  developed,  the  agreements  among 
expert  clinicians  in  both  their  structured  (CRS)  and  un¬ 
structured  ( GRS )  clinical  judgments  were  examined,  and  a 
test  of  concurrent  validity  was  conducted.  The  final  phase. 
Phase  IV,  was  the  evaluation  of  363  consecutive  newborns  by 
non-expert  clinicians  using  the  new  instrument.  From  these 
data,  inter-rater  agreement  on  each  item  in  the  instrument 
was  assessed,  and  patterns  of  responses  to  detect  which 
items  on  the  instrument  were  difficult  to  rate  were  studied. 


This  phase  also  began  the  prospective  validation  of  the  in¬ 
strument. 
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Phase  I ;  Development  of  the  Screening  Instrument 

The  function  of  this  instrument  is  to  screen  newborns 
during  the  postpartum  hospital  stay  for  risk  of  child  mal¬ 
treatment  in  the  future.  This  instrument  is  not  designed 
specifically  as  a  research  tool  to  be  used  on  a  select  popu¬ 
lation.  Rather,  once  fully  tested,  it  is  hoped  that  this 
instrument  will  be  used  on  every  newborn  as  a  first  step  in 
screening;  those  identified  as  at  high  risk  would  then  re¬ 
ceive  further  in-depth  evaluation  by  Social  Services  to  more 
completely  determine  the  type  and  extent  of  preventive  serv¬ 
ices  reguired.  With  this  in  mind,  the  desired  attributes  of 
the  new  instrument  include; 

1)  It  should  be  used  during  the  postpartum  period.  As 
discussed  in  the  last  chapter,  this  is  an  ideal  time 
since  the  family  is  available  for  evaluation,  it  is 
before  any  possible  form  of  maltreatment  could  have 
occurred  to  the  newborn,  it  is  a  time  of  stress  for  the 
family  so  the  parents'  reactions  to  stress  can  be  eval¬ 
uated,  and  evaluations  made  at  this  time  have  been 
shown  to  be  predictive  by  Gray,  Cutler,  Dean,  et  al. 
(1976) . 

2)  It  should  be  simple  and  efficient  to  use  so  that  time 
constraints  will  not  prohibit  clinicians  from  evalu¬ 
ating  all  patients. 
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3)  It  should  not  require  a  special,  formal  interview  or 
self-administered  questionnaire.  Rather,  the  clinician 
should  be  able  to  gather  the  necessary  data  during  the 
course  of  regular  patient  care.  This  will  eliminate 
both  the  problem  of  uncooperative  parents,  a  subpopu¬ 
lation  likely  to  have  many  high  risk  families,  along 
with  the  problem  for  clinicians  of  finding  the  time  to 
do  an  extra  30  to  60  minute  interview  with  each  pa¬ 
tient.  This  requirement  would  also  minimize  the  risk 
of  parents  deliberately  lying  in  response  to  questions 
they  know  are  part  of  an  evaluation  of  them  as  parents. 

4)  It  should  be  fairly  simple  to  apply  and  score.  If 
specially  trained  staff  are  required  to  implement  the 
screening  instrument,  that  would  be  another  obstacle, 
both  in  terms  of  manpower  and  cost,  to  its  widespread 
use . 

5)  Finally,  the  instrument  should  take  the  form  of  struc¬ 
tured  clinical  judgment.  This  format  would  help  ful¬ 
fill  the  preceding  criteria.  This  instrument  could  be 
used  in  the  postpartum  period  since  obstetricians, 
nurses,  pediatricians,  and  social  workers  see  the  fami¬ 
ly  often  at  that  time.  No  special,  time  consuming  in¬ 
terview  would  be  required.  Rather,  the  clinician's 
response  to  each  item  would  be  based  on  observations 
and  data  normally  gathered  regularly  during  the  hospi¬ 
tal  stay.  Also,  since  the  evaluation  is  based  on  the 
entire  stay,  there  is  less  of  a  chance  that  families 


will  be  able  to  lie  or  cover-up  risk  factors. 
Construction  of  the  instrument 
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A  prototype  method  of  how  to  develop  an  observational 
scale  was  demonstrated  by  McCarthy  in  the  development  of  the 
Yale  Observation  Scale  for  determining  level  of  illness  in 
the  febrile  child.  In  McCarthy's  study,  262  febrile  chil¬ 
dren  were  observed  by  pediatricians,  pediatric  residents, 
and  nurses.  The  observers  listed  those  observations  that 
he/she  felt  were  important  in  making  an  overall  assessment 
of  the  child's  severity  of  illness.  The  most  frequently 
mentioned  variables  were  selected  and  from  those  20,  six 
were  found  by  multiple  regression  analysis  to  be  "independ¬ 
ent  predictors  of  serious  illness  (McCarthy,  Sharpe,  Spie- 
sel,  et  al.,  1982)."  These  six  variables  were  then  combined 
into  one  instrument  (McCarthy,  Jekel ,  Stashwick,  et  al., 
1981;  McCarthy,  Sharpe,  Spiesel,  et  al.,  1982). 

McCarthy's  model  of  instrument  development  was  followed 
in  the  development  of  a  screening  instrument  for  risk  of 
child  maltreatment.  A  study  of  residents  at  Yale-New  Haven 
Hospital  ( Y-NHH)  by  Leventhal ,  Fearn,  and  Stashwick  found 
that  pediatric  residents  relied  on  observations  more  than 
the  medical  interview  to  judge  quality  of  parenting.  Impor¬ 
tant  variables  that  went  into  making  this  judgment  included 
how  the  mother  uses  the  medical  system;  observations  of  the 
mother-child  interaction,  the  mother,  and  the  child;  and 
information  from  the  medical  history  (Leventhal,  Fearn, 
Stashwick,  1986).  Altemeier  concurs  that  a  clinician's  sub- 
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jective  impression  is  a  very  good  predictor  of  who  will  or 

will  not  maltreatment  his/her  child.  Altemeier  writes: 

A  major  question  in  our  minds  is  whether ...  subjec¬ 
tive  impressions  could  be  used  in  objective,  con¬ 
sistent,  and  reproducible  fashion  as  part  of  a 
second  generation  interview:  Their  effectiveness 
when  listed  as  specific  observations  rather  than 
open-ended  impressions  remains  to  be  determined 
experimentally  (Altemeier,  O'Connor,  Vietze,  et 
al.,  1984). 

The  original  Leventhal  study  was  later  expanded.  Le- 
venthal,  Garber,  and  Brady  reviewed  the  records  of  every 
infant  referred  during  the  postpartum  period  to  the  hospi¬ 
tal's  child  maltreatment  ( DART-Detection/Assessment/Report- 
ing/Treatment )  committee  in  order  to  study  whether  those 
judged  to  be  at  high  risk  by  unstructured  clinical  judgment 
were  subsequently  maltreated  more  often  than  a  low  risk  non- 
ref  erred  control  group.  In  this  study,  a  list  was  compiled 
of  the  reasons  for  referral  stated  by  clinicians  in  their 
referrals  of  newborns  to  the  DART  committee  (Table  2)  (Le¬ 
venthal,  Garber,  Brady,  1989).  Since  Leventhal 's  earlier 
research  found  that  unstructured  clinical  judgment  could 
identify  those  later  maltreated,  the  list  of  reasons  for 
referrals  of  newborns  was  used  as  the  initial  set  of  risk 
factors  to  be  included  in  the  new  screening  instrument  de¬ 
veloped  in  this  current  study. 

A  similar  list  of  variables  used  in  making  unstructured 
clinical  judgments  about  quality  of  parenting  was  compiled 
by  Ounsted  and  colleagues  (Table  3).  This  list  also  is  a 
mixture  of  observational  variables  and  variables  from  the 
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medical  history.  Like  Leventhal  and  Garber,  Ounsted  includ¬ 
ed  maternal  behaviors  in  the  hospital,  maternal  attitude, 
previous  abuse  of  children,  and  family  conflict  issues 
(Ounsted,  Roberts,  Gordon,  et  al.,  1982). 

An  alternative  method  to  use  in  gathering  variables  to 
include  in  a  screening  instrument  would  have  been  to  select 
those  variables  shown  in  previous  research  to  be  independent 
predictors  of  child  maltreatment.  The  major  problem  with 
this  method,  however,  is  that  a  valid  profile  of  the  child 
abuser  has  yet  to  be  constructed  (Starr,  1987).  In  evaluat¬ 
ing  the  literature  on  risk  factors  for  child  maltreatment, 
it  is  clear  that  there  is  yet  to  be  overwhelming  agreement 
about  which  variables  are  definite  risk  factors  (Table  4). 
Several  possible  reasons  for  the  conflicting  results  of  the 
various  studies  include  differences  in  how  the  outcome 
(child  maltreatment)  was  measured,  the  use  of  different 
types  of  control  groups  (including  the  use  of  unmatched  con¬ 
trol  groups),  and  detection  bias  (Leventhal,  Egerter,  Mur¬ 
phy,  1984). 

Because  of  the  conflicting  results  among  the  various 
studies,  the  current  development  of  a  child  maltreatment 
screening  instrument  started  with  McCarthy's  method  of  in¬ 
cluding  only  those  variables  actually  used  by  clinicians. 

The  risk  factors  considered  for  the  instrument  were  those 
variables  identified  in  the  Leventhal,  Garber,  and  Brady 
study  as  being  used  by  clinicians  in  their  unstructured 
judgments  of  risk.  In  addition,  to  be  included  in  the  in- 
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strument,  rigorous  prior  research  must  support  a  variable  as 
a  risk  factor  for  future  maltreatment.  Further,  in  the 
process  of  testing  the  new  screening  instrument,  only  those 
variables  shown  to  be  obtainable  and  predictive  of  maltreat¬ 
ment  were  to  be  retained. 

Version  1  of  the  newly  developed  screening  instrument 
(the  Clinical  Rating  Scale,  CRS)  is  shown  in  Figure  1.  Each 
of  the  22  variables  was  derived  from  the  Leventhal,  Garber 
and  Brady  study  and  is  expressed  in  a  dichotomous-ordinal 
scale  ranging  from  three  to  five  categories.  Category  1  for 
each  variable  represents  "good"  parenting  while  categories  2 
through  5  represent  worsening  degrees  of  "bad"  parenting. 
Also  included  for  each  item  is  a  box  to  indicate  an  "un¬ 
known"  response. 

The  transformation  of  the  list  of  variables  in  Table  2 
into  the  Clinical  Rating  Scale  instrument  in  Figure  1  was 
accomplished  after  a  careful  review  of  the  literature.  How 
each  variable  has  been  described  in  the  past  (the  range  of 
behaviors  or  traits),  and  the  associated  degree  of  risk  of 
maltreatment  (low,  mild,  moderate,  high)  for  each  specific 
behavior  or  trait  were  studied.  Each  variable  was  then  op¬ 
erationalized  into  a  dichotomous-ordinal  scale.  All  no  risk 
behaviors  or  traits  were  grouped  into  category  1;  thus, 
category  1  versus  all  other  categories  created  the  dichoto¬ 
mous  (Absence  of  Risk  Factor  versus  Presence  of  Risk  Fac¬ 
tor)  nature  of  the  scale.  The  low  to  high  risk  behaviors  or 
traits  were  then  expressed  in  an  ordinal  scale  ranging  from 
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category  2  to  category  5.  All  variables  did  not  readily 
fall  into  a  5-point  scale.  Those  with  only  3  or  4  catego¬ 
ries  thus  had  an  empty  category  somewhere  on  the  scale. 

A  Global  Rating  Scale  (GRS)  was  also  constructed  (Fig¬ 
ure  3).  This  scale  measures  a  clinician's  unstructured 
clinical  judgment  of  a  family's  risk  of  subsequent  mal¬ 
treatment.  This  scale  was  developed  in  order  to  be  able  to 
compare  clinicians'  unstructured  clinical  judgments  to  their 
structured  clinical  judgments  (using  the  Clinical  Rating 
Scale)  of  the  same  families  and  to  test  whether  the  struc¬ 
tured  clinical  judgment  is  an  improvement  in  predictive 
ability  over  the  unstructured  judgment. 

Phase  II :  Pilot  Testing  of  the  Instrument 

The  next  step  in  development  was  to  pilot  test  Version 
1  of  the  newly  developed  Clinical  Rating  Scale. 

Subjects 

There  were  two  classes  of  subjects  in  this  study.  The 
first  class  consisted  of  the  clinicians  (nurses,  pediatric 
residents,  and  social  workers)  who  cared  for  postpartum  wom¬ 
en  and  newborns.  These  clinicians  were  asked  to  evaluate 
the  families  of  newborns  using  both  the  Clinical  Rating 
Scale  and  the  Global  Rating  Scale.  The  clinicians  were  con¬ 
sidered  subjects  since  their  abilities  to  evaluate  families 
were  examined  in  this  study.  The  criteria  for  inclusion 
were  that  the  clinician  a)  must  have  been  a  registered 
nurse,  a  pediatric  resident,  or  a  social  worker  b)  must  have 
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regularly  worked  on  a  postpartum  floor  and  c)  must  have 
cared  for  one  of  the  study's  enrolled  newborns  or  newborn 
families  for  the  duration  of  their  hospital  stay.  Most  fam¬ 
ilies  in  the  study  were  evaluated  by  the  mother's  postpartum 
nurse  and  the  pediatric  resident  who  cared  for  the  baby  in 
the  well-baby  nursery.  Only  those  families  who  were  visited 
by  a  social  worker  as  part  of  their  medical  care  (about  30% 
of  all  postpartum  mothers  who  receive  care  at  Yale-New  Haven 
Hospital's  Women's  Center  are  seen  by  a  social  worker  at  the 
request  of  a  nurse,  obstetrician,  or  pediatrician)  were  also 
evaluated  for  this  study  by  a  social  worker.  Those  babies 
who  were  cared  for  in  the  Newborn  Intensive  Care  Unit  and 
were  never  transferred  to  the  well-baby  nursery  prior  to 
discharge  did  not  receive  a  pediatric  evaluation  because  of 
time  constraints  on  these  residents.  This  amounted  to  less 
than  3%  of  all  births.  Consent  for  participation  in  the 
study  was  obtained  orally  from  the  clinicians  on  a  case  by 
case  basis. 

The  second  class  of  subjects  in  this  study  consisted  of 
the  families  of  newborns  who  were  evaluated  for  risk  of 
child  maltreatment.  Inclusion  criteria  were  that  the  fami¬ 
lies  must  a)  have  delivered  a  viable  infant  at  Yale-New 
Haven  Hospital  b)  have  stayed  for  more  than  48  hours  on  the 
postpartum  floor  (those  mothers  who  went  to  a  surgical  or 
intensive  care  floor  after  delivery  were  excluded)  c)  have 
spoken  English  fluently  d)  have  planned  to  bring  their  new¬ 
born  to  Yale-New  Haven  Hospital's  Primary  Care  Center  for 
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routine  pediatric  care  (this  would  allow  the  medical  records 
of  the  newborns  to  be  reviewed  in  the  future  to  determine  if 
child  maltreatment  had  occurred). 

Families  were  enrolled  in  the  study  if  after  review  of 
the  mother's  and  newborn's  current  hospital  chart  it  was 
determined  that  all  of  the  inclusion  criteria  were  met. 
Consent  was  not  obtained  for  the  following  reason.  Since 
1967 ,  clinicians  at  Yale-New  Haven  Hospital  have  been  obli¬ 
gated  to  report  those  children  they  feel  are  at  high  risk  of 
subsequent  maltreatment  to  the  hospital's  child  abuse 
committee.  This  is  routinely  done  using  unstructured  clini¬ 
cal  judgments  without  the  parents'  knowledge  or  consent. 

This  research  did  not  alter  this  procedure  nor  subject  the 
children  or  their  parents  to  more  scrutiny  than  they  would 
have  normally  experienced.  In  fact,  all  information  ob¬ 
tained  during  the  study  was  strictly  confidential.  The  re¬ 
sponsibility  to  report  children  at  risk  remained  with  the 
clinicians;  none  of  the  information  obtained  in  this  study 
was  used  to  register  children  with  the  child  abuse  commit¬ 
tee  . 

Revision  of  the  instrument 

In  the  pilot,  176  families  were  evaluated  by  their 
clinicians.  For  each  family,  the  mother's  nurse,  the  pedi¬ 
atric  resident,  and  the  social  worker  (if  there  was  one) 
were  asked  to  complete  the  Clinical  Rating  Scale  and  the 
Global  Rating  Scale  based  on  information  they  had  gathered 
during  the  course  of  their  care  of  the  mother  and  infant; 
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the  clinicians  were  asked  not  to  conduct  a  special  interview 
of  the  family.  In  this  way,  the  testing  of  the  instrument 
would  most  simulate  how  the  instrument  would  be  used  if  in¬ 
corporated  into  routine  postpartum  care.  Stapled  to  the  end 
of  each  form  was  a  sheet  asking  the  clinicians  for  their 
feedback  on:  a)  the  variables  selected  (Were  there  some 
they  felt  they  could  never  answer?  Were  there  variables 
they  felt  should  be  added  in?)  b)  the  individual  rating 
scale  for  each  of  the  22  variables  (Were  the  examples  given 
for  each  rating  appropriate  and  clear?  Were  there  better 
examples?  Were  the  categories  helpful?)  c)  how  the  entire 
Clinical  Rating  Scale  was  constructed  d)  any  other  sugges¬ 
tions  . 

During  the  course  of  the  piloting,  the  Global  Rating 
Scale  was  revised  once  (Figure  4).  This  revision  softened 
the  tone  of  the  examples  given  for  each  rating.  In  the 
first  version,  clinicians  were  hesitant  to  give  a  family  a 
rating  of  four,  even  if  they  felt  the  family  was  at  very 
high  risk,  because  the  tone  in  version  1  implied  more  cer¬ 
tainty  than  most  clinicians  were  willing  to  ascribe  to  their 
judgments . 

The  Clinical  Rating  Scale  underwent  five  revisions. 

For  the  most  part,  the  revisions  were  rewordings  of  titles 
or  of  the  examples  in  order  to  clarify  them.  The  section  on 
parental  background  (items  2-9)  was  changed  to  refer  only  to 
the  background  of  the  mother.  The  item  on  maternal  child¬ 
hood  was  divided  into  two  items,  one  on  childhood  stability 
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and  one  on  presence  or  absence  of  maltreatment.  One  ques¬ 
tion  asking  whether  the  father  has  any  of  the  risk  factors 
listed  earlier  on  the  instrument  was  added;  since  the 
clinicians  were  able  to  gather  little  information  on  the 
father,  they  felt  that  one  general  question  was  sufficient. 
If  more  specific  questions  were  asked,  they  would  have  been 
marked  "unknown"  over  80%  of  the  time.  The  item  about  the 
mother's  cooperativeness  in  the  hospital  was  combined  with 
the  item  on  the  mother's  threatening  behavior.  A  shortened 
version  of  the  Global  Rating  Scale  was  added  to  the  end  of 
the  Clinical  Rating  Scale.  By  putting  both  scales  on  the 
same  sheet  the  visual  presentation  of  the  study  was  improved 
and  thus  less  intimidating  to  the  clinicians;  presented 
with  fewer  papers,  clinicians  were  more  apt  to  complete  and 
return  the  forms. 

The  major  revision  of  the  Clinical  Rating  Scale  was  the 
conversion  of  all  the  individual  scales  to  a  four-point 
scale.  The  original  format  that  included  blank  squares  was 
confusing  with  some  clinicians  inappropriately  marking  the 
blank  areas.  The  original  format  also  contained  a  weighting 
to  the  categories.  A  variable  that  only  had  three  catego¬ 
ries  was  given  less  weight  in  the  scoring  (since  the  highest 
category  was  a  three)  than  a  variable  that  had  five  catego¬ 
ries.  Some  items  with  three  categories  had  blank  squares  in 
the  middle  of  the  scale  in  order  to  increase  the  scoring  of 
the  last  categories.  These  weightings  were  originally  made 
without  rigorous  scientific  support.  To  correct  this,  all 
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variables  were  changed  to  a  four-point  scale  with  equal 
weighting  and  no  blank  squares.  If  analysis  of  the  follow¬ 
up  data  reveals  that  certain  items  are  more  predictive  than 
others,  appropriate  weights  will  then  be  added  into  the 
scoring.  Version  6  of  the  Clinical  Rating  Scale  (the  final 
version)  is  shown  in  Figure  2. 

Phase  III :  Tests  of  Inter-Expert  Agreement  and 
Concurrent  Validity 

Since  child  maltreatment  is  the  result  of  a  combination 
of  personality,  environmental,  and  situational  factors,  all 
of  which  are  undergoing  constant  change,  it  is  impossible  to 
predict  with  100%  certainty  who  will  and  who  will  not  mal¬ 
treat  their  children.  There  is  no  absolute  gold  standard  of 
prediction.  However,  as  has  been  discussed  earlier,  it  is 
possible  to  identify  those  who  are  at  high  risk  of  maltreat¬ 
ment.  The  most  predictive  evaluation  would  be  expected  from 
those  experienced  in  working  with  families  where  maltreat¬ 
ment  has  occurred.  By  combining  the  evaluations  of  two  or 
more  such  experts,  a  consensus  evaluation  can  be  derived  and 
used  as  a  gold  standard.  This  method  of  establishing  a  ref¬ 
erence  standard  is  called  consensual  (or  concurrent)  valida¬ 
tion  (Feinstein,  1987). 

The  purpose  of  the  test  of  concurrent  validity  was  to 
compare  non-experts  using  the  Clinical  Rating  Scale  to  the 
gold  standard  (the  level  of  risk  of  child  maltreatment  as¬ 
signed  by  experts  using  unstructured  clinical  judgment. 
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their  GRS )  in  their  abilities  to  differentiate  high  risk 
from  low  risk  newborns.  In  this  part  of  the  study,  the 
agreement  between  pairs  of  expert  clinicians  was  also  stud¬ 
ied.  The  experts  consisted  of  two  pediatricians  and  one 
social  worker,  each  of  whom  has  conducted  research  in  the 
area  of  child  maltreatment  and  has  cared  for  families  that 
have  experienced  maltreatment. 

The  weighted  kappa  statistic  was  planned  to  be  used  to 
compare  item  by  item  agreements  in  risk  ratings  between 
pairs  of  experts.  Cicchetti  has  shown  that  the  minimum  sam¬ 
ple  size  needed  in  order  to  use  kappa  can  be  estimated  by 
2k2  where  k  is  the  number  of  categories  in  a  scale  (Cicchet¬ 
ti,  Sparrow,  1981).  Since  the  Clinical  Rating  Scale  has  4 
categories  for  each  individual  scale,  the  minimum  sample 
size  needed  for  this  part  of  the  study  is  approximately  32. 
Thirty-two  interviews  were  done  by  the  experts,  eleven  by 
each  of  two  possible  pairings  of  the  three  experts,  and  ten 
by  the  third  pairing.  Thus,  each  of  the  three  experts  was 
involved  in  the  evaluation  of  21  or  22  families. 

For  this  phase  of  the  study,  the  experts  evaluated  fam¬ 
ilies  already  enrolled  for  Phase  IV  of  the  study.  So  in 
addition  to  the  experts'  evaluations,  each  postpartum  moth¬ 
er's  nurse,  newborn's  pediatrician,  and  family's  social 
worker  (if  assigned)  were  also  asked  to  complete  the  CRS. 

The  families  to  be  interviewed  were  preselected  to  include 
an  even  balance  of  those  determined  by  the  non-expert  clini¬ 
cians  to  be  high  risk  and  low  risk.  Since  the  overall  na- 
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tional  incidence  of  child  maltreatment  is  estimated  at  less 
than  10%,  this  was  necessary  to  avoid  interviewing  only  low 
risk  families.  The  experts  were  unaware  of  the  risk  status 
assigned  by  the  non-expert  clinicians. 

To  conduct  this  phase  of  the  study,  oral  consent  to 
participate  was  obtained  from  each  mother  interviewed.  The 
experts  explained  the  purpose  of  the  study,  gave  the  mother 
an  information  sheet  that  also  explained  the  study,  and  then 
obtained  oral  consent  prior  to  beginning  the  interview.  In 
each  case,  the  mother  was  interviewed  by  one  of  the  experts 
while  the  second  expert  observed.  Each  interview  lasted 
from  15  to  30  minutes  and,  making  certain  to  cover  the  22 
items  on  the  CRS,  consisted  of  the  medical  history  and  fami¬ 
ly  history  that  the  experts  normally  employ  when  evaluating 
families  in  clinic.  After  the  interview,  the  experts  re¬ 
viewed  the  mother's  prenatal  chart  and  hospital  chart,  and 
spoke  to  the  mother's  nurse.  Then  each  separately  rated  the 
family  using  version  6  of  the  Clinical  Rating  Scale  and  us¬ 
ing  their  unstructured  clinical  judgments.  For  their  un¬ 
structured  judgments,  the  experts  agreed  to  the  definition 
of  each  risk  rating  as  in  Figure  5.  These  definitions  are 
comparable  to  version  2  of  the  GRS. 

Development  of  a  scoring  method 
To  develop  a  scoring  method,  the  instrument  will  have 
to  be  tested  on  a  large  population  of  newborns.  Once  the 
outcomes  for  the  children  are  known  (abuse  or  non-abuse), 
the  instrument  can  undergo  multiple  regression  analysis  to 
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determine  appropriate  weights  for  each  item  on  the  instru¬ 
ment  (Feinstein,  1985,  Ch.  10),  and  the  instrument  can  be 
analyzed  to  determine  the  most  sensitive  and  specific  cut¬ 
off  score.  The  initial  work  of  evaluating  a  large  newborn 
population  with  the  instrument  has  begun  (see  Phase  IV  be¬ 
low).  In  the  meantime,  a  preliminary  scoring  method  was 
developed  from  the  evaluations  completed  by  the  experts  in 
child  maltreatment.  Five  possible  scoring  methods  were  con¬ 
sidered: 

1)  The  arithmetic  sum.  Each  item  on  the  instrument  was 
given  a  value  of  0  if  "unknown"  was  marked  or  a  re¬ 
sponse  was  omitted,  1  if  category  1  (no  risk  trait)  was 
marked,  2  if  category  2  (low  risk  trait)  was  marked,  3 
if  category  3  (moderate  risk  trait)  was  marked,  or  4  if 
category  4  (highest  risk  trait)  was  marked.  The  sum  of 
each  of  the  22  items  was  used  as  the  score.  The  higher 
the  score,  the  more  at  risk  the  subject.  This  method 
was  considered  the  simplest  and  a  good  starting  place. 

2)  The  average  score.  This  score  is  the  result  of  the 
arithmetic  sum  (method  1,  above)  divided  by  the  number 
of  guestions  answered  (that  is,  those  questions  where 
the  response  was  not  omitted  and  was  not  "unknown"). 
This  method  takes  into  account  those  items  left  un¬ 
marked  or  marked  as  "unknown."  Method  1  would  give  a 
higher  score  to  an  evaluation  where  all  22  items  were 
marked  2  (low  risk)  than  to  an  evaluation  where  20 
items  were  marked  2  and  two  were  marked  "unknown." 


Method  2  would  give  both  evaluations  the  same  score, 
thereby  not  "penalizing"  an  evaluation  for  having  fewer 
unknowns . 

The  frequency  of  items  given  a  value  of  2,  3,  or  4. 

This  method  simply  counts  the  number  of  items  where  a 
value  of  2  or  greater  was  assigned.  In  the  dichoto¬ 
mous-ordinal  scale  used  in  the  instrument,  category  1 
indicates  "no  risk"  behaviors  or  traits  while  categor¬ 
ies  2,  3,  and  4  indicate  "at  risk"  behaviors  with  a 
progression  from  "low  risk"  in  category  2  to  "high 
risk"  in  category  4.  This  method  counts  the  frequency 
of  "at  risk"  behaviors  or  traits.  This  method  also 
takes  into  account  items  left  blank  or  marked  "un¬ 
known."  By  method  2,  an  evaluation  with  1  item  marked 
in  category  4  and  22  items  marked  in  category  1  has  a 
lower  score  than  an  evaluation  with  1  item  in  category 
4,  19  items  in  category  1,  and  3  items  marked  unknown. 
Method  3  would  give  each  of  the  two  evalua  tions  the 
same  score. 

The  number  of  items  given  a  value  of  3  or  4 .  This 
method  counts  the  number  of  items  where  a  value  of  3  or 
4,  the  higher  risk  behavior  or  traits,  were  assigned. 
This  method  is  very  similar  to  method  3  but  only  counts 
the  more  "at  risk"  categories. 

The  arithmetic  sum  of  those  items  marked  2  or  higher. 
Like  method  1 ,  this  method  adds  the  values  assigned  to 
each  item,  but  only  if  the  value  is  2  or  higher.  This 
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method  only  counts  behaviors  or  traits  marked  as  "at 
risk."  As  opposed  to  method  3,  this  method  weights  the 
severity  of  the  value  given  to  each  item.  By  method  3, 
an  evaluation  with  1  item  marked  in  category  4,  1  item 
marked  in  category  2,  and  21  items  marked  in  category  1 
has  the  same  score  as  an  evaluation  with  2  items  marked 
in  category  2  and  21  items  marked  in  category  1.  By 
method  5,  the  first  evaluation  would  have  the  higher 
score  since  it  contained  a  more  severe  rating  of  an 
item. 

For  each  of  the  five  possible  scoring  methods,  a  histo¬ 
gram  was  made  of  each  expert's  score  on  the  Clinical  Rating 
Scale  compared  to  the  same  expert's  assessment  using  the 
Global  Rating  Scale  (Figures  6-10).  Based  on  these  graphs, 
it  was  apparent  that  Method  5  produced  the  most  differentia¬ 
tion  between  the  No/Low  Risk  group  and  the  Moderate/High 
Risk  group.  A  cutoff  score  of  9  was  chosen  to  give  a  margin 
of  error  that  erred  on  the  side  of  overestimation  of  the 
number  at  risk.  Since  this  instrument  is  a  screening  tool 
intended  to  lead  to  further  evaluation  of  those  at  risk, 
overestimation  in  order  to  prevent  possible  harm  is  desired 
over  underestimation.  By  this  method,  the  CRS  yields  a  bi¬ 
nary  rating  of  either  High  Risk  or  Low  Risk.  Those  whose 
scores  egualed  or  exceeded  9  were  given  a  CRS  rating  of  High 
Risk  while  those  whose  scores  were  less  than  9  were  given  a 
CRS  rating  of  Low  Risk. 
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Inter-expert  agreement 

Before  the  consensual  validation  could  be  analyzed,  how 


well  the  experts  agreed  in  their  unstructured  clinical  judg¬ 
ments  had  to  be  determined  and  the  gold  standard  estab¬ 
lished.  To  simplify  the  analysis,  the  GRS  ratings  were  di¬ 
chotomized  into  Low  Risk  (GRS  ratings  of  1  or  2)  and  High 
Risk  (GRS  ratings  of  3  or  4).  Four  types  of  inter-rater 
agreements  were  analyzed. 

1)  The  agreement  for  the  32  pairs  of  unstructured  judg¬ 
ments  was  computed  in  several  ways.  Percent  overall 
agreement,  percent  agreement  on  just  the  High/Moderate 
Risk  cases,  and  percent  agreement  on  just  the  Low/No 
Risk  cases  were  computed  first.  The  percent  agreements 
on  High/Moderate  Risk  and  Low/No  Risk  are  proportions 
of  specific  agreement.  They  analyze  separately  the 
agreements  on  the  two  categories  of  risk  and  represent 
the  probability  that  a  second  rater  will  chose  a  spe¬ 
cific  risk  category  if  an  earlier  rater  did  so  already 
( Fleiss ,  1981). 

The  two  experts'  overall  GRS  ratings  for  each  of 
the  32  subjects  agreed  with  each  other  in  29  instances. 
This  GRS  rating  represented  the  gold  standard  of  pre¬ 
diction.  In  the  three  instances  of  disagreement,  the 
higher  rating  was  taken  as  the  gold  standard  (again, 
this  was  done  to  err  on  the  side  of  over-prediction) . 

The  32  pairs  of  unstructured  assessments  were  also 
tested  for  agreement  using  the  kappa  statistic.  Kappa 
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is  a  measure  of  agreement  between  2  observers  that  cor¬ 
rects  for  agreements  expected  by  chance  (Fleiss,  1981, 
Ch.  13).  Kappa  ranges  from  -1  (complete  disagreement) 
to  +1  (complete  agreement)  with  0  representing  chance 
agreement . 

2)  To  test  the  experts'  agreements  using  the  CRS,  the  per¬ 
cent  overall  agreement,  the  percent  agreement  on 
High/Moderate  Risk,  the  percent  agreement  on  Low/No 
Risk,  and  the  kappa  statistic  was  used  to  compare  the 
32  pairs  of  CRS  overall  ratings  of  risk. 

3)  CRS  item  by  item  agreement  among  the  experts  was  then 
examined.  For  each  item,  weighted  kappa  was  calculat¬ 
ed.  While  weighted  kappa  corrects  for  chance-expected 
agreements,  like  kappa  does,  it  also  weights  the  sever¬ 
ity  of  disagreements.  A  dichotomous-ordinal  scale  like 
the  ones  used  on  the  CRS  can  yield  disagreements  in 
both  the  existence  of  a  problem  and  the  severity  of  a 
problem.  Weighted  kappa  takes  into  account  both  possi¬ 
ble  types  of  disagreements  (Fleiss,  1981,  Ch.  13).  The 
weights  used  are  given  in  Figure  11. 

4)  In  addition,  using  each  expert's  own  binary  GRS  rating 
as  the  gold  standard  of  prediction,  the  sensitivity  and 
specificity  of  the  experts'  CRS  ratings  were  calculat¬ 
ed.  Since  the  GRS  rating  served  as  a  standard  of  pre¬ 
diction,  the  sensitivity  and  specificity  evaluated  the 
performance  of  the  CRS  as  a  "diagnostic  test"  (Fleiss, 
1981,  Ch.  1).  Overall  percent  agreement,  percent 
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agreement  on  High/Moderate  Risk  cases,  percent  agree¬ 
ment  on  No/Low  Risk  cases,  and  the  kappa  statistic  for 
agreement  comparing  an  experts'  CRS  rating  to  his/her 
own  GRS  rating  were  done. 

Non-expert  evaluations  versus  the  gold  standard 
The  actual  consensual  validation  took  place  by  compar¬ 
ing  the  non-experts'  CRS  ratings  to  the  gold  standard,  the 
consensus  expert  GRS  rating.  Of  the  32  subjects,  24  had  CRS 
evaluations  returned  by  their  nurses,  20  had  both  CRS  and 
GRS  evaluations  returned  by  their  nurses,  22  had  CRS  evalua¬ 
tions  returned  by  their  pediatricians,  20  had  both  CRS  and 
GRS  evaluations  returned  by  their  pediatricians,  and  only  9 
were  assigned  an  in-hospital  social  worker,  in  every  in¬ 
stance  of  which  both  CRS  and  GRS  evaluations  were  returned. 
The  sensitivity,  specificity,  percent  overall  agreement,  and 
proportions  of  specific  agreement  of  just  the  nurses'  CRS 
evaluations  and  just  the  pediatricians'  CRS  evaluations  were 
calculated.  (Social  work  data  were  not  analyzed  individual¬ 
ly  since  the  number  was  too  small.)  Finally,  the  test  in¬ 
dices  of  the  instrument  using  the  highest  rating  given  by 
any  of  the  3  clinicians  to  a  given  subject  were  calculated. 
For  all  of  these  computations,  only  those  CRS  ratings  that 
had  a  corresponding  GRS  rating  were  used. 

To  examine  whether  the  CRS  improved  the  prediction  of 
the  non-experts,  their  GRS  predictions  were  compared  to 
their  CRS  predictions.  The  sensitivity  and  specificity  of 
both  the  nurses'  GRS  ratings  and  CRS  ratings  when  compared 
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to  the  gold  standard  were  calculated  and  compared.  The  same 
was  done  for  the  pediatric  evaluations  and  for  the  highest 
evaluation  given  each  subject  by  any  non-expert.  An  in¬ 
crease  of  the  CRS  sensitivity  over  the  GRS  sensitivity  would 
mean  that  more  at-risk  subjects  were  identified  with  the  new 
instrument.  Ideally,  an  increase  in  CRS  specificity  over 
GRS  specificity  should  occur.  However,  allowing  for  a  mar¬ 
gin  of  error  that  errs  on  the  side  of  caution  (overestima¬ 
tion  of  those  at  risk)  means  that  the  specificity  would  be 
poorer  than  the  sensitivity.  A  specificity  below  50%  would 
not  be  desired,  though  because  that  would  signal  very  poor 
discrimination  of  those  who  are  not  at  risk. 

Phase  IV :  Evaluation  of  the  Instrument 

This  phase  of  the  study  involved  the  use  of  the  new 
instrument  on  a  large  population  of  newborns.  Non-expert 
clinicians  used  the  instrument  to  evaluate  363  consecutive 
newborns  and  their  families  over  a  one-year  period.  The 
inclusion  criteria  for  both  the  clinicians  and  the  infants 
were  the  same  as  described  for  Phase  II.  In  addition,  the 
mothers'  medical  charts  were  abstracted  to  obtain  informa¬ 
tion  on  demographic  characteristics  (e.g.  age,  race, 
parity) .  The  data  gathered  in  Phase  IV  were  used  to  develop 
a  profile  of  the  study  population,  to  determine  those  items 
about  which  non-experts  had  difficulty  making  judgments,  and 
to  test  inter-rater  agreement  on  the  22  CRS  items. 

The  demographic  profile  of  the  entire  enrolled  study 
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population  was  composed.  Although  363  subjects  were  en¬ 
rolled  in  the  study,  not  all  subjects  had  evaluations  re¬ 
turned.  To  compare  if  the  subpopulation  of  newborns  without 
any  evaluations  differed  from  the  subpopulation  that  had  at 
least  one  evaluation  returned,  the  demographic  profiles  of 
the  two  subpopulations  were  compared.  If  there  were  no  sig¬ 
nificant  differences,  the  chances  were  good  that  the  subpop¬ 
ulation  without  any  evaluations  does  not  differ  much  from 
the  population  that  was  evaluated.  If  there  were  signifi¬ 
cant  differences  in  the  demographic  profiles,  it  is  more 
likely  that  the  unevaluated  subpopulation  is  different  from 
the  rest  of  the  study  group;  most  worrisome  would  be  the 
possibility  that  the  unevaluated  subpopulation  represented 
High  Risk  families  that  went  unrecognized  because  they  were 
able  to  keep  clinicians  from  getting  to  know  them  well. 

Using  the  scoring  method  developed  in  Phase  III,  the 
subpopulation  of  newborns  that  was  evaluated  by  a  clinician 
was  divided  into  High  Risk  and  Low  Risk  categories  based  on 
the  highest  rating  given  to  the  subject  by  any  clinician. 

The  percentage  of  newborns  labeled  High  Risk  was  computed. 

To  learn  which  items  were  most  difficult  for  non-expert 
clinicians  to  assess,  the  percentage  of  times  each  item  was 
marked  "unknown"  or  left  blank  was  calculated  for  each  CRS 
item  for  each  type  of  clinician.  A  ranked  ordering  of  items 
by  most  difficult  to  least  difficult  was  compared  among  the 
three  types  of  clinicians.  This  ranked  ordering  of  the  non¬ 
experts'  was  then  compared  to  a  ranked  ordering  of  the  items 
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based  on  the  experts'  percentage  of  unknowns  for  each  item. 

The  final  analysis  of  this  study  was  to  test  item  by 
item  agreement  among  the  clinicians.  For  all  of  the  infants 
where  an  evaluation  was  returned  by  both  a  nurse  and  a  pe¬ 
diatrician,  the  weighted  kappa  statistic  was  calculated  for 
each  of  the  22  items  on  the  Clinical  Rating  Scale  to  deter¬ 
mine  the  level  of  agreement  between  nurse  and  pediatrician. 
The  process  was  repeated  on  the  evaluations  where  both  the 
nurse  and  a  social  worker  returned  evaluations  and  then 
again  on  the  evaluations  where  both  the  pediatrician  and  a 
social  worker  returned  evaluations.  High  inter-rater  agree¬ 
ment  would  signal  that  the  three  types  of  clinicians  were 
consistently  evaluating  families  using  the  same  set  of  stan¬ 
dards.  The  purpose  of  these  analyses  was  to  determine  if 
the  evaluations  from  the  three  types  of  clinicians  could  be 
pooled  to  render  one  rating  of  risk.  Pooling  of  data  would 
be  desirable  because  of  the  high  number  of  "unknowns"  marked 
on  individual  evaluations.  By  pooling  the  data,  these  un¬ 
knowns  could  be  reduced.  In  routine  use  of  the  CRS,  the 
ability  to  pool  data  means  that  different  types  of  clini¬ 
cians  could  collaborate  in  the  evaluation  of  a  family. 

In  the  future,  these  data  will  be  part  of  a  study  of 
the  predictive  validity  of  the  instrument.  When  the  new¬ 
borns  have  passed  their  second  birthdays,  and  again  after 
their  fourth  birthdays,  their  medical  records  will  be  re¬ 
viewed  to  determine  if  maltreatment  has  occurred  in  the  in- 
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tervening  years.  From  these  data,  the  predictive  ability  of 
the  Clinical  Rating  Scale  will  be  measured  and  the  length  of 
optimal  prediction  of  the  instrument  will  be  determined. 
Because  of  changing  family,  social,  and  psychological 
states,  it  is  expected  that  the  instrument  will  be  maximally 
predictive  up  to  the  second  year  of  life.  As  discussed  ear¬ 
lier,  these  data  will  also  allow  the  scoring  method  to  be 
refined. 


CHAPTER  FOUR 


RESULTS 
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Phases  I  and  II 

The  results  of  Phase  I  and  Phase  II  can  be  found  in 
Figures  1  through  4 .  These  two  phases  involved  the  develop¬ 
ment  and  refinement  of  both  the  Global  Rating  Scale  ( GRS , 
overall  assessment  based  on  unstructured  clinical  judgment) 
and  the  Clinical  Rating  Scale  (CRS,  assessment  based  on 
structured  clinical  judgment).  Version  6  of  the  CRS  (Figure 
2)  incorporated  the  final  GRS  into  the  final  CRS  and  was  the 
version  used  in  Phases  III  and  IV. 

Phase  III 

The  families  of  32  newborns  were  evaluated  by  both  ex¬ 
pert  and  non-expert  clinicians  in  this  phase  of  the  study. 
While  all  32  were  evaluated  by  two  experts  using  both  the 
GRS  and  the  CRS,  the  number  of  non-expert  evaluations  varied 
from  0  to  3 .  There  was  also  a  tendency  for  the  non-experts 
to  not  complete  the  GRS  portion  of  the  evaluation.  Only  6 
subjects  were  evaluated  on  the  CRS  by  all  three  non-experts 
and  2  of  these  subjects  were  missing  one  of  the  GRS  evalua¬ 
tions;  15  subjects  were  evaluated  on  the  CRS  by  only  2  non¬ 
experts,  and  3  of  these  were  missing  one  GRS;  7  subjects 
were  evaluated  on  the  CRS  by  only  1  non-expert,  and  1  was 
missing  the  GRS;  4  subjects  were  not  evaluated  on  the  CRS 
by  any  non-experts.  Thus,  of  the  32  subjects,  28  had  at 
least  one  CRS  evaluation,  and  27  had  at  least  one  complete 
GRS  and  CRS  evaluation,  for  a  pooled  total  of  55  CRS's  re- 
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turned  and  49  CRS/GRS's  turned  in. 

Development  of  a  scoring  method 

As  discussed  in  the  Methods  section,  the  preliminary 
scoring  system  for  the  CRS  was  developed  by  analyzing  the 
expert  evaluations  of  the  32  subjects.  The  histograms  in 
Figures  6  through  10  each  represent  the  five  different  scor¬ 
ing  methods  proposed.  Each  histogram  groups  each  expert's 
CRS  scores  by  the  same  expert's  GRS  ratings  of  risk.  The 
best  scoring  method  would  yield  the  least  overlap  of  No/Low 
Risk  subjects  to  Moderate/High  Risk  subjects  both  within 
each  rater  and  among  all  three  raters. 

Figure  6,  the  arithmetic  sum,  was  the  starting  place. 
For  experts  2  and  3,  there  was  too  much  overlap  of  the  Mod¬ 
erate/  High  Risk  group  with  the  No/Low  Risk  group.  Figure 
7,  the  average  score,  improved  on  this  by  clearly  separating 
out  the  High  Risk  group  from  the  Low  Risk  group  of  expert  3. 
However,  the  degree  of  overlap  in  expert  2  worsened,  and  the 
2  risk  groups,  while  still  separated,  were  brought  closer 
for  expert  1.  Figure  8,  the  frequency  of  items  marked  2  or 
higher,  was  a  further  improvement  by  clearly  separating  out 
the  No/Low  Risk  group  for  expert  2,  keeping  the  2  groups 
separated  for  expert  1,  and  shrinking  the  amount  of  overlap 
for  expert  3.  Figure  9,  the  frequency  of  items  marked  3  or 
higher,  did  not  change  expert  3's  groupings,  but  increased 
the  overlap  of  experts  1  and  2  and  was  clearly  a  step  back¬ 
wards  as  far  as  discriminating  between  the  two  risk  groups. 
Figure  10,  the  arithmetic  sum  of  those  items  marked  2  or 
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higher  provided  the  best  resolution  of  the  two  risk  groups. 
The  two  groups  were  distinct  for  experts  1  and  2  with  a  very 
small  overlap  for  expert  3.  In  addition,  this  method  main¬ 
tained  the  wide  separation  between  groups  for  expert  1. 

This  method  also  made  intuitive  sense.  Rather  than 
just  count  the  number  of  items  marked  with  some  degree  of 
risk  (ratings  2,  3,  and  4),  this  method  weights  those  items 
so  that  an  infant  with  2  high  risk  ratings  (4's)  gets  a 
higher  risk  score  than  an  infant  with  2  low  risk  ratings 
(2's).  While  a  cut-off  score  of  10  rendered  all  of  the  Mod¬ 
erate/High  Risk  subjects  (as  judged  by  the  experts'  GRS  rat¬ 
ings)  classified  as  High  Risk  by  the  CRS,  a  score  of  9  or 
higher  was  chosen  as  the  cut-off.  A  margin  of  error  was 
built  in  to  decrease  the  chance  of  missing  an  infant  at 
risk.  As  a  first  step  screening  tool,  the  preference  is  to 
over-predict  rather  than  miss  an  infant  in  need  of  interven¬ 
tion  . 

Inter-expert  agreement 

1)  When  the  agreement  on  the  pairs  of  experts'  GRS  ratings 
was  examined  (Table  5),  the  percent  observed  agreement 
(Po)  was  91%  and  kappa  was  0.80.  The  proportions  of 
specific  agreement  were  88%  for  High/Moderate  Risk 
cases  and  92%  for  Low/No  Risk  cases.  This  is  consid¬ 
ered  excellent  inter-rater  agreement  (Cicchetti,  Spar¬ 
row,  1981 ) . 

Since  the  composition  of  each  of  the  32  pairs  of 
experts  could  be  any  of  three  possible  combinations, 
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the  assignment  of  which  expert  in  each  pair  was  labeled 
"Expert  1"  and  which  was  labeled  "Expert  2"  for  the 
contingency  table  was  random.  This  is  acceptable  so 
long  as  the  number  of  disagreements  between  the  experts 
remained  small.  As  can  be  seen  in  Table  5,  there  were 
only  3  disagreements.  Changing  the  assignment  of  which 
expert  was  "Expert  1"  and  which  was  "Expert  2"  would 
not  change  Po  and  would  change  kappa  by  no  more  than 
0.01 . 

2)  Table  6  shows  the  experts'  contingency  table  of  CRS 
ratings.  Again,  the  assignment  of  which  expert  in  each 
of  the  32  pairs  was  labeled  "Expert  1"  and  which  was 
labeled  "Expert  2"  was  random.  The  Pq  for  the  overall 
risk  rating  was  84%,  agreement  on  High/Moderate  Risk 
cases  was  88%,  agreement  on  Low/No  Risk  cases  was  76%, 
and  kappa  was  0.65,  a  level  considered  good  (Cicchetti, 
Sparrow,  1981). 

3)  The  experts'  item  by  item  agreements  were  calculated 
using  Pq  and  weighted  kappa.  The  results,  ranked  in 
order  from  most  to  least  agreement  is  shown  in  Table  7. 
A  Po  of  greater  than  or  egual  to  0.70  and  a  kappa  of 
greater  than  or  equal  to  0.40  is  considered  fair  agree¬ 
ment  (Volkmar,  Cicchetti,  Dykens,  et  al.,  1988).  All 
but  two  of  the  items  met  these  requirements  for  fair 
agreement.  The  two  items  that  did  not  meet  the  crite¬ 
ria  for  fair  agreement  had  low  kappas  but  high  (86%) 
observed  overall  agreements.  The  reason  for  the  low 
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kappa  is  that  the  expected  agreements  (Pe)  on  these  two 
questions  were  also  very  high  (greater  than  80%); 
since  kappa  considers  and  adjusts  for  agreements  ex¬ 
pected  by  chance,  a  high  Po  coupled  with  a  high  Pe 
yields  a  low  kappa  (Cicchetti,  1988). 

4)  The  contingency  table  of  each  expert's  CRS  rating  com¬ 
pared  to  his/her  own  GRS  rating  for  each  subject  is 
shown  in  Table  8.  Using  the  GRS  as  the  predictive 
standard,  the  evaluation  by  the  CRS  as  a  diagnostic 
test  had  a  sensitivity  of  100%  and  a  specificity  of 
51%.  This  indicates  that  the  CRS  overestimates  the 
number  at  high  risk;  all  of  the  errors  occurred  in  the 
direction  of  misidentifying  Low  Risk  subjects  as  High 
Risk  subjects.  As  discussed  above,  this  was  intention¬ 
ally  done  in  the  process  of  developing  a  scoring  meth¬ 
od.  When  intra-rater  agreement  (each  expert's  CRS  rat¬ 
ing  compared  to  his/her  own  GRS  rating) ,  was  calculat¬ 
ed,  Po=70%  and  kappa=0 . 45 ,  fulfilling  the  criteria  for 
fair  agreement.  The  proportion  of  specific  agreement 
for  High/Moderate  Risk  cases  was  72%  compared  to  68% 
for  Low/No  Risk  cases.  Thus,  the  agreement  was  higher 
for  those  subjects  felt  to  be  at  High/Moderate  Risk. 
Non-expert  evaluations  versus  the  gold  standard 
The  results  of  the  consensual  validation  are  shown  on 
Tables  9  through  14.  The  nurses'  GRS  evaluations  (Table  9) 
when  compared  to  the  gold  standard  had  a  sensitivity  of  40% 
and  a  specificity  of  100%  meaning  that  the  nurses'  unstruc- 
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tured  judgments  overestimated  the  number  not  at  risk  and 
misidentif ied  the  majority  of  those  subjects  the  experts 
felt  were  at  risk.  The  nurses'  CRS  evaluations  (Table  10) 
had  a  sensitivity  of  64%  and  a  specificity  of  0.78%.  The 
CRS  thus  improved  the  nurses'  identification  of  those  at 
risk.  This  occurred  with  a  decrease  in  specificity,  yet 
without  lowering  the  specificity  to  an  unacceptably  low  lev¬ 
el.  The  CRS  also  improved  the  nurses'  proportion  of  specif¬ 
ic  agreement,  raising  it  from  57%  to  70%.  This  means  that 
if  the  experts  felt  a  subject  was  at  High/Moderate  Risk, 
there  was  a  57%  probability  that  the  nurses  would  concur 
using  their  unstructured  judgments  but  a  70%  probability 
that  the  nurses  would  concur  if  they  used  the  CRS  to  help 
them  in  their  evaluation. 

The  pediatricians'  GRS  evaluations  (Table  11)  had  a 
sensitivity  of  63%,  a  specificity  of  92%,  and  a  kappa  of 
0.57.  The  use  of  the  CRS  (Table  12)  did  not  change  any  of 
the  test  indices  for  the  pediatricians.  This  indicates  that 
the  non-expert  pediatricians  did  a  good  job  of  discriminat¬ 
ing  who  is  at  risk  of  child  maltreatment  with  and  without 
the  Clinical  Rating  Scale. 

Table  13  compares  the  highest  GRS  rating  given  to  each 
subject  by  a  non-expert  (social  work  data  included)  to  the 
gold  standard.  The  sensitivity  was  54%  and  the  specificity 
was  86%.  As  shown  in  Table  14,  the  highest  CRS  rating  given 
to  each  subject  by  a  non-expert  had  a  sensitivity  of  77%,  a 
specificity  of  79%,  a  kappa  of  0.55,  and  a  proportion  of 
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specific  agreement  of  the  High/Moderate  Risk  cases  of  77%. 
Overall,  these  indices  are  better  than  for  the  nurse  evalua¬ 
tions  considered  singly  or  the  pediatrician  evaluations  con¬ 
sidered  singly. 

In  summary,  experts  using  their  unstructured  clinical 
judgments  have  good  inter-rater  agreement  in  assessing  an 
infant's  risk  of  subsequent  child  maltreatment.  Using  the 
newly  developed  Clinical  Rating  Scale,  the  experts  maintain 
good  agreement  with  each  other  and  with  their  own  original 
unstructured  evaluations  of  risk.  Using  the  experts'  un¬ 
structured  judgments  as  the  standard  of  prediction,  the  CRS 
has  high  test  indices  when  used  by  non-experts  and  shows 
promise  as  a  screening  instrument  for  newborns. 

Phase  IV 

Of  the  363  subjects  enrolled  for  this  phase,  26  (7%) 
had  no  evaluations  returned  by  a  non-expert  nurse,  pediatri¬ 
cian,  or  social  worker.  Table  15  shows  the  distribution  of 
returns . 

Five  of  the  363  (1%)  maternal  medical  charts  were  una¬ 
vailable  for  review.  Table  16  gives  a  demographic  profile 
of  the  entire  population,  the  subpopulation  that  had  at 
least  one  evaluation  returned,  and  the  subpopulation  that 
had  no  evaluation  returned.  The  2  subpopulations  had  simi¬ 
lar  profiles.  There  were  some  minor  differences  between  the 
2  groups,  but  since  one  subpopulation  is  23  times  the  size 
of  the  other  (N=337  vs.  N=26)  it  is  difficult  to  draw  the 
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conclusion  that  there  is  a  significant  difference.  The  fact 
that  the  2  subpopulations  are  fairly  similar  in  race,  gender 
of  the  infant,  type  of  delivery,  and  marital  status  signal 
that  the  two  subpopulations  are  probably  identical. 

In  Phase  IV,  the  subjects  were  given  a  rating  of  High 
Risk  or  Low  Risk  based  on  the  CRS  by  each  clinician  who  re¬ 
turned  a  CRS.  Table  17  gives  a  profile  of  how  each  type  of 
clinician  rated  the  population.  The  social  work  evaluations 
differed  from  those  of  the  nurses  and  pediatricians  in  that 
the  social  workers  did  not  evaluate  all  of  the  subjects; 
due  to  limited  resources,  the  social  workers  only  visited 
those  patients  referred  by  nurses  and  doctors.  Thus,  it  is 
expected  that  a  higher  percentage  of  this  group  would  be 
identified  as  high  risk  as  compared  to  the  entire  population 
seen  by  the  nurses  and  pediatricians. 

To  study  how  difficult  certain  items  were  for  clini¬ 
cians  to  evaluate,  the  percentage  of  times  each  item  was 
left  blank  or  marked  "unknown"  by  each  type  of  clinician  was 
computed  and  is  shown  in  Table  18.  Table  19  shows  the  same 
data  but  ranks  the  questions  from  highest  to  lowest  percent¬ 
age  of  unknowns.  The  non-expert  clinicians  were  similar  in 
which  items  were  more  difficult  although,  not  surprisingly, 
the  individual  percentages  of  unknowns  were  lower  for  the 
social  workers  than  for  the  other  types  of  clinicians.  For 
the  most  part,  the  difficult  questions  concerned  background 
information  about  the  mothers  (e.g.  history  of  violence, 
criminal  record,  psychiatric  history,  care  as  a  child,  and 
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family  conflict)  while  the  least  difficult  questions  con¬ 
cerned  in-hospital  behaviors  of  the  mother. 

Of  note,  the  item  on  past  care  of  children,  an  item  one 
would  intuitively  expect  to  be  an  indicator  of  risk  of  mal¬ 
treatment,  was  one  clinicians  were  reluctant  to  evaluate. 

It  is  unclear  whether  clinicians  had  difficulty  talking  to 
mothers  about  child  care  and  their  past  care  of  children  or 
whether  they  had  difficulty  judging  the  honesty  of  the  re¬ 
sponses  obtained.  That  the  experts  marked  "unknown"  up  to 
23.4%  of  the  time  (mode=4.7%,  median=4 . 7% ,  only  2  items  were 
never  marked  unknown)  despite  interviewing  the  mothers  with 
the  CRS  items  in  mind,  indicated  that  difficulty  in  forming 
judgments  about  people  played  a  significant  role  in  the  high 
percentages  of  "unknowns"  marked. 

Since  each  group  of  clinicians  evaluated  a  different 
number  of  subjects,  it  is  possible  that  the  rankings  would 
be  different  if  the  exact  same  subjects  were  evaluated  by 
each  type  of  clinician.  To  test  this,  Table  20  ranks  the 
items  again  by  percentage  of  unknowns  but  only  for  the  190 
patients  evaluated  by  both  a  nurse  and  a  pediatrician.  This 
table  is  very  similar  to  Table  19  and  suggests  that  Table  19 
is  a  fairly  accurate  reflection  of  how  much  difficulty  each 
type  of  clinician  had  with  each  item. 

The  final  question  was  how  well  the  non-experts  agreed 
with  each  other  on  each  item  of  the  CRS.  Tables  21,  22,  and 
23  list  the  weighted  kappa,  observed  percent  agreement  (Po), 
and  expected  percent  agreement  (Pe)  for  each  item  for  each 
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of  the  three  possible  pairings  of  non-experts.  The  items 
are  ranked  from  highest  to  lowest  weighted  kappa  values. 

Only  5  of  the  22  items  met  the  criteria  for  fair  agreement 
( Pq>  0.70  and  weighted  kappa>0.40).  The  other  items  yielded 
a  low  weighted  kappa  in  the  face  of  high  Po  values.  As  oc¬ 
curred  when  the  experts7  item  by  item  agreements  were  exam¬ 
ined,  the  low  weighted  kappas  resulted  from  very  high 
chance-expected  agreements .  For  the  most  part  on  these 
items,  subjects  were  rated  as  No  Risk  or  Low  Risk,  ratings 
the  majority  of  the  population  would  be  expected  to  receive. 
Since  kappa  evaluates  agreements  correcting  for  chance,  on 
items  where  there  is  a  low  prior  probability  of  being  rated 
Moderate/High  risk  (and,  thus,  a  high  PJ  ,  kappa  will  be  low 
even  if  the  observed  agreement  is  very  high. 

Tables  21,  22,  and  23  show  that  despite  low  values  for 
weighted  kappa,  the  non-expert  clinicians  had  excellent  ob¬ 
served  agreements  for  the  vast  majority  of  the  items.  This 
indicates  that  the  non-expert  clinicians  completing  the  CRS 
evaluations  individually  were  using  similar  standards  to 
rate  subjects.  This  further  indicates  that  the  data  from 
the  different  types  of  clinicians  could  be  pooled  in  order 
to  decrease  the  number  of  unknowns  on  a  subject  before  a 
final  score  was  tallied.  Another  application  of  this  analy¬ 
sis  would  be  to  have  the  different  types  of  clinicians  con¬ 
fer  with  each  other  and  complete  the  CRS  as  a  joint  effort. 
The  item  by  item  agreement  analysis  indicates  that  complet- 
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ing  the  CRS  by  consensus  would  not  lead  to  conflicting  con¬ 
clusions  about  a  subject's  risk  for  each  item. 


CHAPTER  FIVE 


DISCUSSION 
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Prior  studies  have  shown  that  expert  and  non-expert 
clinicians  can  identify  those  at  risk  of  child  maltreatment. 
Global  scales  of  unstructured  clinical  judgment  are  simple, 
direct,  and  easy  to  construct  and  use.  These  global  scales, 
however,  have  unspecified  components  with  unspecified 
demarcations  (Feinstein,  1987).  Such  vagueness  leads  to 
wide  variances  in  evaluations  among  different  clinicians  and 
inconsistencies  within  clinicians. 

This  study  standardized  judgments  about  a  newborn's 
risk  of  subsequent  maltreatment  and  thus  improved  the  abili¬ 
ty  of  clinicians  to  identify  those  families  who  would  most 
benefit  from  social  service  support.  This  was  accomplished 
by  first  identifying  the  risk  factors  that  experienced 
clinicians  evaluate  in  making  their  judgments  about  a 
child's  risk  of  subsequent  maltreatment.  The  measurement  of 
each  of  these  risk  factors  was  then  standardized.  The  re¬ 
sulting  Clinical  Rating  Scale  (CRS)  was  pilot  tested,  re¬ 
fined,  and  then  tested  on  a  large  population  of  newborns. 
This  study  has  shown  that  the  CRS  has  good  inter-rater  and 
intra-rater  reliability.  Used  as  part  of  the  routine  care 
of  all  newborns,  the  CRS  should  help  non-expert  clinicians 
focus  their  evaluations  and  use  specific  criteria  in  their 
judgments . 

Strengths  and  Limitations 

The  CRS  is  practical  for  use  as  a  screening  tool  on  all 
newborns.  As  discussed  in  Chapter  Two,  the  ideal  screening 


tool  would  be  used  in  the  postpartum  period,  would  use 
structured  clinical  judgment,  and  would  not  require  a  spe¬ 
cially  designed  interview.  The  CRS  meets  all  of  these  crit¬ 
eria.  Because  it  can  be  completed  expeditiously,  is  simple 
to  use,  and  does  not  require  special  training  for  clini¬ 
cians,  it  has  strong  potential  for  widespread  clinical  ap¬ 
plication  . 

The  consensual  validation  revealed  good  sensitivity  and 
specificity  of  the  CRS  as  a  screening  test  when  compared  to 
the  current  gold  standard  of  prediction.  There  was  overall 
improvement  of  the  non-experts'  judgments  over  their  un¬ 
structured  clinical  judgments.  In  particular,  the  judgments 
of  nurses  were  significantly  improved.  There  was  also  good 
inter-rater  agreements  on  the  22  items  in  the  rating  scale 
indicating  consistency  in  the  use  of  the  instrument  by  dif¬ 
ferent  evaluators. 

The  CRS  improves  on  prior  instruments  that  screen  for 
risk  of  maltreatment  in  several  respects.  The  CRS  takes 
less  than  five  minutes  to  complete  at  the  end  of  a  mother's 
postpartum  hospital  stay.  The  instruments  of  Monaghan  and 
colleagues  (Monaghan,  Gilmore,  Muir,  et  al . ,  1986)  and  Alte- 
meier  and  colleagues  (Altemeier,  O'Connor,  Vietze,  et  al., 
1984)  both  required  interviews  of  about  one  hour.  Altemei¬ 
er  's  interview  yielded  a  sensitivity  of  65%  compared  to  a 
sensitivity  of  64%  for  nurses  using  the  CRS  and  63%  for  pe¬ 
diatricians  using  the  CRS.  Browne  developed  a  12-item  post¬ 
partum  checklist  that  was  reported  to  have  a  sensitivity  of 
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81%  and  a  specificity  of  94%  but  a  positive  predictive  accu¬ 
racy  of  only  7%  (Browne,  Lowton,  1987;  Browne,  Saqi,  1988). 
This  low  positive  predictive  accuracy  resulted  in  part  be¬ 
cause  of  a  low  prior  probability.  Whether  the  CRS  can  im¬ 
prove  on  this  positive  predictive  accuracy  awaits  the  longi¬ 
tudinal  follow-up  phase.  However,  it  is  expected  that  there 
will  be  a  higher  positive  predictive  accuracy  because  the 
population  in  this  study  has  a  higher  incidence  of  child 
maltreatment. 

A  major  problem  in  this  study  was  the  high  percentage 
of  times  that  items  were  marked  "unknown"  by  the  non-expert 
clinicians.  Phase  IV  showed  that  clinicians  (especially  the 
nurses  and  pediatric  residents)  had  difficulty  gathering 
sensitive  data  and  making  judgments  about  the  mother's  past. 
Other  studies  on  the  self-report  of  drug  use  have  found  that 
23%  of  illicit  drug  users  deny  their  use  when  questioned 
(Zuckerman,  Amaro,  Cabral,  1989).  In  any  type  of  evalua¬ 
tion,  judging  the  veracity  of  the  data  obtained  is  diffi¬ 
cult  . 

Improvement  of  this  data  gathering  would  require  a  two- 
tiered  approach.  First,  clinicians  need  to  be  encouraged  to 
broach  sensitive  subjects  with  their  patients.  Clinical 
training  needs  to  stress  the  importance  of  past  actions  and 
behaviors  on  the  current/future  health  and  welfare  of  the 
patient  and  his/her  family.  Clinicians  are  in  the  unique  to 
position  of  being  able  to  help  patients  identify  and  address 
areas  of  their  lives  that  threaten  their  health  (e.g.  drug/ 
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alcohol  use,  violent  behavior).  Clinicians  thus  need  to  be 
taught  that  it  is  part  of  their  care  of  patients  to  ask 
these  questions.  This  type  of  increased  experience  at  gath¬ 
ering  sensitive  information  would  make  it  easier  for  clini¬ 
cians  to  ask  these  questions.  Second,  clinicians  need  to 
hone  their  abilities  to  judge  the  quality  of  the  information 
that  they  do  gather.  Increased  experience  asking  sensitive 
questions  would  help  here  also.  Improving  the  quality  of 
the  relationship  between  clinician  and  patient  would  also  be 
a  significant  aid.  The  non-expert  clinicians  in  this  study 
all  had  a  very  short  amount  of  time  (from  two  to  five  days) 
in  which  to  get  to  know  the  families  studied.  That  they 
were  not  completely  comfortable  evaluating  these  families  is 
understandable.  However,  they  had  the  same  amount  of  time 
in  which  to  get  to  know  these  families  as  do  the  clinicians 
who  routinely  care  for  these  families  in  the  postpartum  pe¬ 
riod  and  who  routinely  judge  the  safety  of  sending  a  newborn 
home  with  his/her  parents.  A  system  of  continuity  of  health 
care  would  improve  on  the  clinician-patient  relationship. 
This  issue  is  important  for  the  entire  practice  of  medicine. 
Such  an  improved  relationship  would  help  clinicians  both 
evaluate  risk  factors  and  quickly  help  those  judged  to  be  at 
risk. 

Another  issue  is  the  high  percentage  of  families  iden¬ 
tified  as  High  Risk  by  the  CRS.  The  nurses  and  pediatri¬ 
cians  identified  32%  and  36%  of  the  population  as  being  at 
high  risk.  When  the  highest  rating  given  by  any  clinician 


was  used,  50%  of  the  population  was  identified  as  at  high 
risk.  One  reason  for  this  was  that  the  cut-off  score  was 
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chosen  to  be  lower  in  order  to  over-estimate  the  number  at 
risk.  Since  the  CRS  is  designed  to  be  a  first  step  screen¬ 
ing  tool  with  those  identified  as  High  Risk  going  on  to  more 
in-depth  social  service  evaluation,  it  is  better  to  err  on 
the  side  of  safety.  If  the  cut-off  score  were  to  be  raised 
one  point  to  a  score  of  10,  nurses  would  identify  27%  of  the 
population  as  at  high  risk,  pediatricians  would  identify  32% 
as  such,  and  the  use  of  the  highest  rating  would  identify 
45%  of  the  population  as  at  high  risk. 

Another  possible  reason  for  the  high  number  of  families 
identified  as  High  Risk  is  that  the  individual  risk  factors 
are  not  weighted.  All  of  the  CRS  items  are  counted  as 
egually  important  despite  the  fact  that,  intuitively,  past 
abuse  of  children  would  seem  to  put  a  family  more  at  risk 
than  would  cooperativeness  with  the  hospital  staff.  The 
results  of  the  prospective  study  will  be  necessary  in  order 
to  refine  the  scoring  system. 

Finally,  it  is  difficult  to  say  whether  these  high  per¬ 
centages  reflect  to  some  degree  the  nature  of  the  population 
included  in  this  study.  As  discussed  in  the  Background  sec¬ 
tion  of  Chapter  Two,  estimates  of  the  lifetime  prevalence  of 
child  maltreatment  range  from  10  to  40%.  Until  the  longitu¬ 
dinal  follow-up  of  the  population  is  conducted,  it  is  not 
possible  to  identify  how  much  the  true  characteristics  of 
the  specific  population  studied  contributed  to  this  high 
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percentage . 

Ethical  Considerations 

There  are  several  ethical  issues  entangled  with  the 
concept  of  screening  families  for  risk  of  child  maltreat¬ 
ment.  The  first  is  the  question  of  whether  screening  should 
be  voluntary  or  mandatory.  The  whole  purpose  of  screening 
is  to  protect  children,  yet  they  would  not  be  the  ones  to 
give  consent,  their  parents  would.  Parents  who  are  at  high 
risk  are  less  likely  to  volunteer  for  such  screening.  With 
mandatory  screening,  a  conflict  then  arises  between  the  par¬ 
ents'  right  to  (or  desire  for)  privacy  and  freedom  in  par¬ 
enting  and  the  children's  right  to  life  and  physical  integ¬ 
rity.  Regardless  of  the  choice  that  society  makes  between 
these  two  conflicting  liberties  on  purely  ethical  grounds, 
society  benefits  from  preventing  maltreatment  because  of  the 
high  cost  to  society  of  child  maltreatment  (see  Chapter 
One)  . 

Another  major  area  of  concern  is  that  of  labeling  peo¬ 
ple.  Although  it  is  not  meant  to  be,  being  identified  as 
"High  Risk"  may  be  equated  with  actually  being  a  child  mal- 
treater.  This  labeling  is  even  more  harmful  for  those  who 
are  mislabeled.  With  an  incidence  of  10%,  and  a  sensitivity 
and  specificity  of  75%,  fully  25%  of  the  population  will  be 
misclassif ied  when  compared  to  actual  outcomes  of  parenting. 
For  each  high  risk  family  correctly  identified,  three  low 
risk  families  will  be  misidentif ied  as  being  at  high  risk 


87 


(Kotelchuck,  1982;  Avison,  Turner,  Noh,  1986).  Even  with 
very  sensitive  and  specific  instruments  the  magnitude  of 
misclassif ication  will  still  be  large. 

The  best  ways  to  protect  families  from  being  stigma¬ 
tized  include; 

1)  the  careful  protection  of  confidentiality. 

2)  the  careful  use  of  terms.  Families  should  be  recog¬ 
nized  as  those  who  would  most  benefit  from  social  serv¬ 
ices  and  should  not  be  called  "abusive"  or  "failures." 

3)  an  avoidance  of  coercion.  Consent  and  full  disclosure 
should  be  part  of  all  programs.  Without  evidence  of 
actual  or  imminent  danger  to  the  children,  help  for 
these  families  should  be  voluntary  (Brody,  Gaiss, 

1976) . 

The  large  number  of  misclassif ications  also  includes  a 
large  number  of  false  negatives.  This  misidentif ication  can 
lead  to  a  false  sense  of  security  and  to  ultimate  harm  com¬ 
ing  to  a  child  (Brody,  Gaiss,  1976).  However,  this  is  a 
weak  argument  against  screening.  Without  any  screening  pro¬ 
gram  in  place,  the  situation  that  occurs  in  many  health  cen¬ 
ters,  all  children  at  risk  will  lack  preventive  measures  and 
even  more  children  would  meet  harm. 

Restricting  or  prohibiting  screening  programs  would  not 
ameliorate  all  of  the  issues  associated  with  misidentif ica¬ 
tion  of  families.  As  Daniel  points  out,  most  health  centers 
already  perform  some  sort  of  screening,  all  of  which  are 
associated  with  false  positive  and  false  negative  events 
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(Daniel,  Newberger,  Reed,  1978).  The  screening  in  these 
programs  often  consists  of  informal,  unstructured  clinical 
judgment  that  results  from  routine  clinic  visits.  As  the 
current  study  has  shown,  such  unstructured  judgments  have 
poorer  test  indices  than  structured  instruments  like  the 
CRS.  Not  implementing  structured  screening  programs  would 
ultimately  yield  more  mislabeled  families  than  the  use  of 
the  CRS. 

Conclusions 

Phase  III  evaluated  the  use  of  the  CRS  in  the  hands  of 
both  expert  and  non-expert  clinicians  when  compared  to  the 
current  gold  standard  of  prediction.  When  compared  to  the 
gold  standard,  the  CRS  had  a  good  level  of  sensitivity  and 
specificity  for  the  non-experts  as  well  as  the  experts. 

When  compared  to  actual  outcomes  (whether  the  subjects 
are  actually  maltreated),  it  is  anticipated  that  the  CRS 
will  retain  a  good  level  for  these  test  indices.  However, 
it  is  unlikely  that  either  the  sensitivity  or  the  specifici¬ 
ty  will  exceed  90%  for  many  reasons.  First,  there  is  no 
"typical"  abuser,  so  all  High  Risk  cases  will  not  be  identi¬ 
fied  at  birth  (Roberts,  1988).  Second,  social  pediatric 
illnesses,  (e.g.  trauma,  failure  to  thrive,  accidents,  in¬ 
gestions)  are  on  a  continuum.  There  are  not  always  cut-and- 
dried  demarcations  between  maltreatment  and  non-maltreatment 
(Kotelchuck,  1982).  Also,  issues  related  to  potential  for 
sexual  abuse  are  more  complex  than  can  be  assessed  in  a  rou- 
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tine  clinical  relationship.  Thus,  the  CRS  does  not  attempt 
to  gauge  those  issues  although  sexual  abuse  would  be  counted 
in  any  follow-up  that  identified  which  subjects  were  subse¬ 
quently  maltreated.  Further,  not  all  patients  identified  as 
being  at  high  risk  will  have  a  negative  outcome  since  child 
maltreatment  is  a  result  of  interactions  between  many  per¬ 
sonal  and  environmental  factors.  Schneider  et  al.  found 
that  20%  of  the  population  have  attitudes  and  experiences 
similar  to  those  of  known  abusers,  and  20%  of  the  population 
have  unusual  child  rearing  practices,  yet  most  of  these  peo¬ 
ple  will  not  maltreat  their  children  (Schneider,  Hoffmeis- 
ter,  Heifer,  1976).  Finally,  since  child  maltreatment  is 
the  result  of  many  factors,  this  instrument  is  not  neces¬ 
sarily  looking  for  those  who  will  maltreat  their  children 
but  those  who  are  at  risk  of  maltreatment.  Those  found  to 
be  at  high  risk  would  benefit  from  social  service  interven¬ 
tions.  This  over  prediction  in  the  interest  of  prevention 
leads  to  a  low  specificity. 

Phase  IV  evaluated  the  use  of  the  instrument  by  non¬ 
expert  clinicians.  This  evaluation  demonstrated  that  the 
CRS  is  used  similarly  by  the  different  types  of  clinicians. 
The  three  types  of  clinicians  who  were  studied  had  difficul¬ 
ty  collecting  data  and  making  judgments  on  the  same  set  of 
items.  On  those  items  that  they  did  collect  data  and  make 
judgments,  the  clinicians  had  good  inter-rater  agreement. 
These  results  suggest  that  the  information  from  the  differ¬ 
ent  types  of  clinicians  could  be  pooled  in  completing  the 
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Sensitivity 

<  70% 
70%— 79% 
80%-89% 
90%-100% 


CRS ,  thus  decreasing  the  number  of  unknown  items. 

The  predictive  validity  of  the  CRS  remains  to  be  test¬ 
ed.  However ,  once  these  data  are  collected,  it  will  still 
be  difficult  to  assess  how  well  the  instrument  has  performed 
since  there  have  yet  to  be  established  in  the  literature 
standards  for  screening  instruments.  As  a  first  step  in  the 
establishment  of  such  standards,  the  following  criteria  are 
proposed: 

False  Negative  Error  Clinical  Evaluation 
>30%  Poor 

21%-30%  Fair 

ll%-20%  Good 

0%-10%  Excellent 

The  above  criteria,  coupled  with  a  specificity  of  at  least 
50%,  are  suggested  as  the  minimum  acceptable  test  indices 
for  a  screening  instrument. 

With  most  screening  instruments,  those  who  exceed  a 
specified  cut-off  score  will  be  further  evaluated  for  the 
trait/disease  being  sought.  The  false  positives  on  a  test 
represent  those  who  will  erroneously  have  further,  in-depth 
evaluations  performed.  The  false  negatives  represent  those 
who  will  "pass"  the  screen  and  will,  thus,  receive  no  fur¬ 
ther  evaluation.  While  it  is  desirable  to  minimize  the 
false  positives,  the  commission  of  this  error  has  less  of  a 
negative  outcome  than  the  commission  of  false  negative  er¬ 
ror.  Those  who  pass  the  screen,  but  should  not  have,  will 
receive  no  further  evaluation  and  may  conseguently  suffer 
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physical  harm/illness.  The  above  criteria  thus  emphasize 
the  false  negative  error  and  set  as  the  limit  of  acceptable 
missed  cases  30%.  A  higher  rate  of  missed  cases  would,  in 
practical  terms,  mean  the  screening  instrument  is  ineffec¬ 
tive  in  identifying  the  true  positives. 

A  specificity  of  at  least  50%  is  desirable  because  this 
index  indicates  how  many  of  the  true  negatives  are  correctly 
identified.  If  the  specificity  were  less  than  50%,  then 
those  who  are  truly  without  the  trait/disease  being  sought 
would  have  a  higher  probability  of  being  falsely  identified 
as  positive  than  correctly  identified  as  negative.  While 
the  only  resulting  harm  would  be  that  too  many  subjects 
would  undergo  further,  in-depth  evaluations,  such  a  situa¬ 
tion  would  mean  that  the  screening  instrument  is  less  useful 
in  identifying  the  true  negatives  than  a  flip  of  a  coin. 

Whether  the  CRS  meets  these  criteria  when  compared  to 
the  outcomes  of  the  subjects  remains  to  be  seen.  When  the 
evaluation  of  the  experts  is  used  as  the  standard  of  meas¬ 
urement,  the  CRS  (highest  rating  given  by  an  evaluator.  Ta¬ 
ble  14)  meets  the  criteria  for  fair  evaluation. 

The  initial  evaluation  using  the  CRS  on  363  subjects 
was  completed  in  Phase  IV;  the  long-term  follow-up  of  these 
subjects  remains  to  be  conducted.  From  these  data,  multiple 
regression  analysis  will  be  performed,  and  each  item  in  the 
scale  will  be  weighted  to  reflect  its  importance  on  overall 
predictive  ability.  With  appropriate  weights,  the  scoring 
of  the  instrument  will  be  refined  which  will  improve  the 
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sensitivity  and  specificity  of  the  CRS.  The  regression 
analysis  may  even  lead  to  a  new  method  of  scoring.  Rather 
than  using  an  arithmetic  sum.  Boolean  clusters  or  a  hierar¬ 
chical  system  might  prove  to  be  a  better  scoring  system 
(refer  to  "Clinimetrics  and  the  Development  of  Rating  Scales 
in  Chapter  Two). 

Those  found  to  be  at  risk  of  abuse  should  be  closely 
followed  for  psychosocial  stressors  that  may  lead  to  parent¬ 
ing  failure.  Like  prior  attempts  at  identification  of  those 
at  high  risk,  it  is  expected  that  the  CRS  will  be  predictive 
for  2  to  4  years  with  a  decline  in  predictive  ability  over 
time  (U.S.  Dept.  HHS . ,  1988;  Altemeier,  O'Connor,  Vietze, 
et  al.,  1982).  This  is  because  the  parents  will  change  over 
time  and  because  over  the  years,  social  and  personal  circum¬ 
stances  will  change. 

No  matter  what  the  results  of  the  predictive  validation 
study,  the  CRS  needs  to  be  retested  on  a  new  sample  of  pa¬ 
tients.  Despite  the  large  size  of  the  sample  population 
used  in  Phase  IV,  the  sample  may  not  reflect  the  total  popu¬ 
lation  at  large  (Soper,  Cicchetti,  Satz,  et  al . ,  1988).  A 
difference  in  prevalence  of  maltreatment  between  the  sample 
population  and  the  total  population  can  greatly  affect  the 
test  indices  of  sensitivity  and  specificity. 

As  discussed  in  Chapter  One,  perinatal  screening  for 
risk  of  maltreatment  is  important;  despite  intervention 
after  abuse  has  occurred,  even  after  just  one  event,  the 
maltreatment  can  lead  to  permanent  physical  and  psychologi- 
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cal  damage  to  the  child. 

The  way  forward  for  us  seems  to  be  further  development 
of  screening  programmes  at  the  time  of  birth  when  vul¬ 
nerable  parents  can  be  given  appropriate  help  before 
any  serious  damage,  physical  or  emotional,  has  been 
inflicted  on  the  baby.  Our  ultimate  aim  must  be  pre¬ 
vention  (Lynch,  Roberts,  1982). 


Along  with  perinatal  screening,  ongoing  assessments  of 
families  also  must  occur.  Since  all  people  change  over 
time,  those  who  are  found  initially  to  be  at  low  risk  should 
not  be  overlooked.  They  should  also  be  observed  by  clini¬ 
cians  during  routine  health  care  for  changes  in  personal  or 
social  circumstances  or  for  stressful  life  events  that  may 
turn  a  Low  Risk  family  into  a  High  Risk  family. 

Because  of  our  complex  dynamic  society  with  such  fac¬ 
tors  as  unemployment,  inflation,  infidelity,  etc.,  we 
feel  that  the  child  and  parent  should  be  observed  on  a 
continuing  basis  in  order  that  the  potential  for  child 
abuse  might  be  detected  whenever  it  develops  (Rosen¬ 
berg,  Myers,  Shackleton,  1982). 


The  ability  to  be  a  good  parent  is  partly  innate  and 
partly  a  skill  developed  by  the  experience  of  trial  and  er¬ 
ror.  Just  because  a  parent,  especially  a  new  parent,  does 
not  show  complete  confidence  or  finesse  at  the  birth  of  a 
child,  it  does  not  portend  a  fateful  outcome  for  the  child. 
Thus,  the  CRS  does  not  examine  parenting  style  and  does  not 
seek  to  have  parents  prove  that  they  will  be  good  parents. 
Rather,  the  CRS  assesses  risk  factors  that  could  cause  even 
good  parents  to  experience  parenting  failure.  The  purpose 
of  the  CRS  is  to  identify  those  who  would  benefit  most  from 


scarce  social  service  resources  so  that  their  parenting 
abilities  can  be  maximized. 


FIGURES 


Figure  1 

Parenting  Skills  Screening  Instrument  -  Version  1 
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CARE  OF  (LEO  SEES 
Past  history 

Sibs  uell  cared 
for 

Sib  unkempt,  miss 
school  often,  or 
miss  aDDts.  often 

Documented  hx  of 
Fai  lur  e-tcr-Thri  ve 

Suspicion  of 

Atxise  or  Neglect 

Dxurented  abuse 
>r  neglect 

Unable  to  score 
question/ 

1  hknown 

PARENIS 

Intelligence 

No  concern — both 
parents  able  to 
function  well 

Parents  slew- 
unable  to  inter¬ 
act  wall 

Dbcunented  in 
chart  that  one 
parent  has  ICK70 

Unable  to  score 
question/ 

Unlncxvn 

Psychiatric  Hx 

No  pnor  hx  or 
current  sx 

‘'hid  illness 
(mood  d/o) — under 
control  and  tx 

'bid  illness 
(ncod  d/o) — not 
Icing  tx'ed 

Hijor  illness 
( psychosis, 
iRrsonaiitv  d/o) 

Unable  to  score 
question/ 

Unknown 

Illicit  Drug  Use 

Never  used  drugs 

<2  per  week  pnor 
to  pregnancy — 

none  now 

>2  (cr  week  pnor 
to  pregnancy — 

none  now 

Multiple  drugs  ir 
past  or  any  use 
during  pregnancy 

Unable  to  score 
question/ 

Unkncxvn 

Alcohol  Use 

'"'ever  used 
ilcotwl 

<2  drinks  per  wk 
prior  to  preg. — 
none  now 

>2  drinks  |x.‘r  vk 
prior  to  pr^;. — 

IN  *W 

known  alcohol ic 
or  Hmnk  during 

Pi  ngninev 

Unable  to  score 
question/ 

Unknown 

Criminal  Record 

No  prior 
criminal  record 

Us  been  chirgod 
>2  of  cnrrrs — 
never  convicted 

Currently  Ivts 
d urges  pending 

Ills  been  con- 

victed/jail 

record 

Unable  to  score 
question/ 

Unknown 

Violent  Personalit 

No  violent  verlnl 
or  physical 
outbursts 

Us  violent 
ver  La  1  outbursts- 
never  physical 

Us  rarr  violent 
verbal  and  physi¬ 
cal  outbursts 

lUs  frequent  (>2 
|er  mo. )  physical 
outbursts 

His  in  pin'd  sair.- 
one  enough  to 
rpnui  re  hnsn. 

Unable  to  score 
question/ 

Unknown 

Parents'  Childhood 

lappy  nrmories 
of  childhood 

changed  guardians 
>1 — always  stayed 
with  fanrilv 

Was  in  faster 

care 

Ovinged  foster 

flOTCS 

Was  abused  or 
neglecicd 

Unable  to  score 
question/ 

Unknown 

Maternal  Age 

bther  >18  y.o. 
at  birth  of  first 
child 

bther  now  >18 
but  was  <18  at 
Dirth  of  1st  kid 

'bther  now  <18 

Unable  to  score 
question/ 

Unknown 

CURRENT  FAMILY 

Family  Violence 

No  family 

vi  ol  ence-dciTEStic 

peace 

Uequent  (>2/mo) 
litter  (yelling) 
arpuaents 

Physical 
violence  among 
family  members 

Unable  to  score 
question/ 

Ihknown 

Lifestyle 

Stable — family 
intact 

Fanil  y  nrmbers 
(not  parents) 
move  in/out  often 

No  father 
figure 

'•bther ’s  boy¬ 
friend  changes 
often 

Children  shuttled 
between  different 
farnilv  members 

Unable  to  score 
question/ 

Ihknown 

Housing 

lafe,  clean  home 
with  adequate  rocr 
— move  <l/yr 

bves  >l/yr,  but 
between  safe, 
roomy  hares 

Live  in  unsafe, 

overcrowded 

hone 

Live  in  temporary 
shelters 

Live  on  street/ 
abandoned  bldgs. 

Unable  to  score 
question/ 

1  hknown 

Provisions  for  Eab' 

lave  necessary 
clothes  and 
furniture  ready 

lave  crib,  car 
seat,  baby  food — 
seeking  others 

Actively  seeking 
necessary  items — 
can  borrow  items 

Unprepared — 
parents  unaware 
of  needs 

Unable  to  score 
question/ 

1  hknown 

Social  Supports 

aoncone  can  care 
for  baby  at  all 
Limes 

Jsu.  uimeone  to 
care  for  baby — 
no  one  in  aierg. 

1  rcquently  (>l/wk 
tires  with  no  one 
for  babv  care 

No  provisions  for 
baby  care  on  a 
daily  basis 

Unable  to  score 
question/ 

Unknown 

CURRENT  PRETHANCY 
Attitude  Towards 

teg.  planned 
ind  wanted 

Unplanned  preg. 
but  baby  wonted 

Unplanned  preg. — 
abortion/adoption 
considered 

Parents  remain 
ambivalent 
tovnrds  bnbv 

Unable  to  score 
question/ 

linkmm 

Prenatal  Care 

Regular  care; 
began  <8  weeks 

First  appt  >9wks 
or  missed  _>  3 
annrs. 

[■bther  had  no 
prenatal  care  but 
delivered  in  hosn 

No  prenatal  care 
and  delivered  at 
hemp 

Unable  to  score 
question/ 

Unknown 

CURRENT  BEHAVIORS 
Care  of  Newborn 

'bther  is  atten¬ 
tive  to  baby's 
needs; is  eentle 

•bther  is  atten¬ 
tive  but  handles 
baby  rcxirhlv 

[bther  ignores 
baby's  needs  but 
interacts 

I'bther  ignores 
baby 

Unable  to  score 
question/ 

Unknown 

Visiting  Child 

bther  with  child 
constantly 

bther  with  child 
at  least  1/day 

'bther  visits  <1/ 
day  or  only  when 
pnrrx  imped  to 

[•father  refuses  oi 
reluctant  to  see 
child 

Unable  to  score 
question/ 

1 hknown 

Cooperativeness 

bther  cooperates 
with  hospital 
staff 

[•father  needs  to 
be  coaxed  into 
cooperating 

[■bther  resists 
medical  care  or 
advice 

[■father  tries  to 
leave  hasp,  with 
bebv  AMA 

Unable  to  score 
question/ 

I hknown 

Obstructive 

Behavior 

'bther  cooperates 
with  baby's  med. 
rare 

fbtlrcr  argues  wit 
staff  caring  for 
babv 

f bther  obstructs 
medical  care 
for  babv 

Unable  to  score 
question/ 

1  hknown 

Danger  to  Child 

’  bther  mindful 
of  baby's  safety 

[bther  not 
careful  with  baby 
— unintentional 

fbther  places 
baby  in  obvious 
danger 

fbther  harms 
baby 

Unable  to  score 
question/ 

1  hknoi  .n 

Threats 

'  bther  pleasant 
with  hosp.  staff 

fbtlrer  amhiva- 
lait  tex-irds 
stiff 

[father  argues 
with  staff 

i'btlier  threatens 
staff  with 
physical  linrm 

fbther  tries  to 
actually  harm 
staf  f 

Unable  to  score 
question/ 

Itnknown - 

Ver .  1 
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Figure  Z 

Parenting  Skills  Screening  Instrument  -  Version  6 

For  each  of  the  numbered  items  on  the  left  nlencp  mart  „  . 

th‘"  — k  the  b"'  *’ 


:are  of  older  sic 

Past  History 

1. 

irTVTTTFE — - - 

Un- 

<nown 

5rimip 

No  problems  with  care 

Poor  well-child  core 

(e.g.  missed  appts.) 

i  f/flfift  /  L/U  I 

Suspicion  of  abu90  or 
neglect 

Verified  abuse,  neglect, 
or  failure  to  thrive 

Intelligence 

2. 

Unknown 

No  concerns;  functions 

well 

Slow;  limited  thinking 

Mildly  retarded 

Moderately/severely 

retarded 

Psychiatric  (t) 

3.  History 

Unknown 

No  prior  history  or 
current  symptoms 

Mild  (e.g.  anxiety) 

Past  f  hosp.  admission 
or  current  V  drug  use 

Major  (psychosis,  major 
depression,  suicide  try) 

Illicit  Drug  Use 
4. 

Unknown 

No  history  of  drug  use 

Used  before  pregnancy, 
none  during  pregnancy 

Used  occasionally 
during  pregnancy 

Used  regularly  during 
pregnancy 

Alcohol  Use 

5. 

Unknown 

No  history  of  alcohol 
use 

Drank  before  pregnancy, 
none  during  pregnancy 

Drank  occasionally 
during  pregnancy 

Drank  res.  during  preg. 
or  untreated  alcoholic 

Motivation 

6. 

Unknown 

Uses  resources;  seeks 
help  as  needed 

Seeks  help  once 
encouraged  to  do  so 

Slow  to  address 
problems 

Denies  problems; 
resists  help 

Criminal  Record 

7. 

Unknown 

No  criminal  record 

Has  been  charged  but 
never  convicted 

Currently  has  charges 
pending 

Has  been  convicted  OR 
Jail  stay  now  or  in  past 

History  of 

8.  Violence 

Unknown 

Verbal  outbursts  only 

Rare  physical  outbursts 

Frequent  phvsical 
outbursts 

A  past  victim  has 
needed  medical  care 

Care  as  a  Child 

9. 

Unknown 

No  history  of  abuse 

of  neglect 

Poor  nurturance  (e.g. 
freq.  change  in  care) 

History  of  abuse  or 
neglect 

Spent  time  in  foster 
care 

Age 

10.  i 

Unknown 

>18  at  birth  of  first 

child 

Now  >18,  but  was  <18 
at  birth  of  first  child 

Now  _<  1 8  years  old  , 
but  >15  years  old 

Now  £15  years  old 

LUKKLNI  EAE1NLK 

Gen.  Character 

11. 

Unknown 

'/o  concerns — is 

appropriate 

No  man  is  involved 

Man  involved  has  past 
prob.  in  any  areas  #2-10 

Man  involved  has  current 
prob.  in  any  areas  #'2-10 

tURCENl  lAhULY 

Family  Conflict 

12. 

Unknown 

None  or  mild 

Discord  with  some 
bitter  arguing 

Strong  discord  but  no 
violence 

Physical  violence 
in  family 

Stability  of 

13.  Couple 

Un¬ 

known 

No 

part¬ 

ner 

Stable — couple/family 
intact 

Couple's  relationship 
unstable 

Boyfriends  change 
often 

Absence  of  stable 
relationships 

Housing 

14. 

Unknown 

Safe  and  adequate 

Unsafe  or  overcrowded 
home 

Live  in  temporary 
shelter 

Live  on  street/ 
abandoned  buildings 

Provisions  for 

15.  Baby 

Unknown 

lave  necessities 
ready 

Have  food,  crib,  car 
seat;  need  the  rest 

Still  seeking 
necessities 

Parents  unaware  of 
needs  for  baby 

Social  Supports 

16. 

Unknown 

Family  or  friends 
available  regularly 

Family  or  friends 
available  occasionally 

Family/f riends  avail, 
only  in  emergency 

No  family/f riends  avail 
to  help,  ever 

CURRENT  PREGNANCY 
Attitude 

17. 

Unknown 

Pregnancy  planned  and 
wanted 

Unplanned  pregnancy, 
but  baby  wanted 

Abortion  or  adoption 
was  considered 

Remain  ambivalent  (e.g. 
drug  induced  delivery) 

Prenatal  Care 

18. 

Unknown 

Regular  and  began  in 
the  first  trimester 

Regular  end  began  in 
the  second  trimester 

Began  in  third  trim. 

OR  irreg.  attendance 

No  prenatal  care 

tUkkEHi'  BEHAVIORS 

Care  of  Newborn 
19. 

Unknown 

Mom  attentive  and 
appropriate  with  baby 

Attentive  but  rough 
with  baby 

Ignores  baby's  needs 
often  but  interacts 

Usually  ignores  baby  OR 
interacts  seldom 

Visiting  with 

20.  Baby 

Unknown 

Mom  w/  baby  constantly; 
daily  if  in  NBSCU 

Mom  w/  baby  >$  the  day; 
every  1-2  days  if  NBSCU 

Visits  <i  the  day;l-2 
times  a  week  if  NBSCU 

Reluctant  or  refuses 
to  see  baby 

Cooperativeness 

21. 

Unknown 

Cooperates  with 
hospital  staff 

Resists  medical  care 
or  advice 

Tries  to  leave  with 
baby  AMA 

Threatens  or  tries  to 
harm  staff 

Danger  to  Child 
22. 

Unknown 

Mom  mindful  of  baby's 
safety 

Unintentionally  not 
careful 

Places  baby  in  obvious 
danger  OR  not  careful 

Mother  harms  baby 

Please  complete  the  following  statement  by  circling  the  phrase  in  bold  which  you  feel  is  most  accurate . 

I  think  that  this  baby  is  at  HIGH  RISK  /  MODERATE  RISK  /  LOW  RISK  /  NO  RISK  for  being  abused  or  neglected  sometime 
in  the  future. 


Version  6 
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Figure  3. 

Global  Rating  Scale  -  Version  1 

Based  on  your  clinical  judgment  and  your  interactions  with 

this  newborn's  family,  please  mark  the  one  statement  below 

which  you  feel  is  most  accurate. 

_  I  am  fairly  certain  that  this  child  will  be  abused 

or  neglected  and  needs  to  be  protected  by 
separation  from  the  parents. 

_  I  am  concerned  that  this  child  might  be  abused  or 

neglected,  and  I  feel  that  this  family  will  need 
to  be  closely  followed  and  given  assistance  by 
Social  Services. 

_  I  feel  that  there's  a  chance  this  child  will  be 

abused  or  neglected,  but  I  don't  feel  that 
intervention  is  warranted  at  this  time. 

_  I  feel  that  this  child  will  not  be  abused  or 

neglected . 
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Figure  A 

Global  Rating  Scale  -  Version  2 

Based  on  your  clinical  judgment  and  your  interactions  with 

this  newborn's  family,  please  mark  the  one  statement  below 

which  you  feel  is  most  accurate. 

_  I  am  fairly  certain  that  this  child  will  be  abused 

or  neglected;  this  family  definitely  needs 
assistance  from  Social  Services  (e.g.  parenting 
classes,  support  groups,  assistance). 

_  I  am  concerned  that  this  child  might  be  abused  or 

neglected,  and  I  feel  that  this  family  will  need 
some  assistance  from  Social  Services. 

_  I  feel  that  there's  a  chance  this  child  will  be 

abused  or  neglected,  but  I  don't  feel  that 
intervention  is  warranted  at  this  time. 

_  I  feel  that  this  child  will  not  be  abused  or 

neglected. 
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Figure  5 

Definitions  of  Risk  Ratings 

High  Risk — There  is  a  strong  possibility  the  baby  will  be 
maltreated.  During  the  hospital  stay,  the  parents  have 
exhibited  significant  risk  factors.  The  family  needs  defi¬ 
nite  social  work  evaluation,  a  DART  referral,  and  some  sort 
of  intervention. 

Moderate  Risk — There  is  the  possibility  that  this  baby  will 
be  maltreated.  The  parents  have  revealed  risk  factors 
during  the  hospital  stay,  but  they  have  support  systems 
intact  in  the  family.  A  social  work  evaluation  is  in  order 
and  careful  social  work  follow-up  after  discharge  is 
necessary . 

Low  Risk — The  rater  feels  that  the  baby  is  safe,  but  there 
is  a  slight  possibility  of  maltreatment.  While  no  social 
work  follow-up  or  intervention  is  mandatory  at  this  time, 
the  pediatrician  should  follow  this  family  closely. 

No  Risk — A  strong  feeling  that  the  baby  is  not  at  risk  for 
maltreatment.  The  evaluator  feels  there  are  no  outstanding 
risk  factors  and  feels  comfortable  with  this  rating. 


Figure  6 

Scoring  Method  1:  Sum  of  All  Items 
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High  risk 

k 

Moderate  risk 

Expert  3 

Low  ri: 

No  risl 

Sum  of  Ratings  of  All  CRS  Items 


Figure  7 

Scoring  Method  2:  Average  Score  of  All  Items 
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Expert  2 


No  ri;  k 


Low  risk 


Moderate  risk 


Expert  1 


High  risk 

Moderate  risk 


Low  risk 

No  risk 

— i - 1 - 

1.5 

Average  Rating  of  All  CRS  Items 


4. 


1.0 


2.0 
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Figure  £ 

Scoring  Method  3:  Frequency  of  Ratings  2  or  Higher 


Expert  3 


Expert  2 


No 


Lcxv  risk 


isk 


Moderate  risk 


Expert  1 


5  10 

Number  of  2,  3,  or  4  Ratings 


Figure  9 

Scoring  Method  4:  Frequency  of  Ratings  3  or  Higher 
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Expert  3 


Moderate  risk 

Low 

risk 

No  r 

isk 

High  risk 

Moderate 

risk 

Expert  2 

Low 

risk 

No  r 

isk 

Expert 


High  risk 


Moderate  risk 
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Eijgure  10 

Scoring  Method  5:  Sum  of  Items  Rated  2  or  Higher 
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Figure  11 

Weights  Used  in  Analysis  of  CRS 
Item  By  Item  Agreements 
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Table  1 

A  Comparison  of  the  Contents  of  Instruments 
that  Screen  for  Risk  of  Child  Maltreatment 
(Key  to  studies  at  end  of  table) 


Study 

Item  1.  2.  3.  4.  5.  6.  7.  8.  9.10.11 


l.Past  care  of  children 

* 

* 

* 

* 

2. Mom's  intelligence 

* 

* 

3. Mom's  psychiatric 

history 

* 

* 

* 

* 

* 

* 

* 

4. Mom's  drug  use 

* 

* 

5. Mom's  alcohol  use 

* 

6. Mom's  motivation  to 

solve  own  problems 

* 

* 

* 

7. Mom's  criminal  record 

* 

* 

8. Mom's  history  of 

violence 

* 

* 

* 

* 

9. Mom's  care  as  a  child 

* 

* 

* 

* 

* 

* 

10. Mom's  age 

* 

* 

* 

* 

* 

* 

11. Father's  character 

* 

12. Family  conflict 

* 

* 

* 

13. Family  stability 

* 

* 

* 

14 . Housing 

* 

* 

* 

* 

15 . Provisions  for  baby 

* 

16. Social  supports 

* 

* 

* 

* 

17. Attitude  towards 

current  pregnancy 

* 

* 

* 

* 

* 

* 

18. Prenatal  care 

* 

* 

* 

19. Care  of  newborn 

* 

* 

20. Visiting  with  baby 

* 

* 

* 

21 . Cooperativeness 

* 

* 

* 

* 

22. Danger  to  child 

* 

23. Observed  abusive 

tendency 

* 

24. New  infant  in  family 

* 

* 

25 . Truthfulness 

* 

26. Job  change 

* 

* 

27. Marital  status 

* 

* 

* 

* 

* 

28. Parenting  judgment 

* 

* 

* 

* 

29 . Perspective  on  life 

* 

30 . Self-esteem 

* 

* 

* 

* 

31. Ability  to  handle 

stress 

* 

* 

32. >1  child  £5  y.o. 

* 

* 

33 . Depression 

* 

* 

34. Multiple  crises  or 

stresses 

* 

* 

* 

35 .Unrealistic  expecta- 

tions  of  child 

* 

* 

36. Child  perceived  as 

difficult 

* 

* 

( continued) 
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Table  1 .  continued 


_ Item _ 

3 7.  Time  between 
pregnancies 

38 .  Post-delivery 
separation 

3 9.  Prematurity,  low 
birthweight , 
handicap 

40.SES  problems 

41 . Socialization 
difficulties 

42. Social  work 

consult  at  delivery 

4 3.  Mom  concerned  with 
parenting  ability 

44 . Neighborhood 

45.  Mom  rigid  or 
domineering 

46.  Poor  self- 
under  standing 

47. Observed  violence 
as  a  child 

48. Parenting  skills 

49 . Inconsistent  use 
of  discipline 

50.  Deficit  in 
parenting  knowledge 

51 .  Parent/child  person¬ 
ality  mismatch 

52.  Child  resembles 
disliked  adult 

53 . Reconstituted 
family 

54.  Many  children 

55.  Death  of  friend/ 
family 

56. Sudden  loss/chronic 
illness 

57. Tolerance  of  physical 
punishment 

58. Sexual  stereotypes 
of  children 

59.  Violence  in  media 

60.  Extreme  ideas  of 
rights  or  privacy 

61.  Prenatal  classes 

62 . Smoking 

63.  Breast  feeding 

64 .  Pregnancy/delivery 
complications 
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Key  to  Studies  in  Table  1 

1.  Kung/Leventhal  Parenting  Skills  Instrument 

2.  Altemeier,  O'Connor,  Vietze,  et  al.  (1984) 

3.  Murphy,  Orkow,  Nicola  (1985) 

4.  Monaghan,  Gilmore,  Muir,  et  al.  (1986) 

5.  Avison,  Turner,  Noh  (1986) 

6.  Browne,  Lowton  (1987) 

7.  Lynch,  Roberts  (1978) 

8.  Garbarino,  Sherman  (1980) 

9.  Daro  (1988) 

10.  Murphy,  Jenkins,  Newcombe,  et  al.  (1981) 

11.  Lealman,  Haigh,  Phillips,  et  al.  (1983) 
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Table  2 

Reasons  for  Referrals  to  a  Child  Abuse  Registry 
(From  Leventhal,  Garber,  Brady,  1989) 

Categories  Reasons  for  Referrals 

Care  of  Previous  Children  Sibling  abused 

Sibling  neglected/f ailure-to- 
thrive/history  of  poor  child 
care 

Characteristics  of  Parents  Mental  retardation 

Serious  psychiatric  problem 

Drug  abuse 

Alcohol  abuse 

Jail  history 

History  of  violence 

Abused/neglected  as  a  child 

Adolescent  parent 

Current  Family  Violence  in  family 

Chaotic  lifestyle 
Inadequate/overcrowded  housing 
No  provisions  for  baby  at  home 
No  social  support 

Current  Pregnancy  Unwanted/adoption  considered/ 

strongly  ambivalent 
No  prenatal  care/delivery  at 
home 

Current  Behaviors  Parents  (mother)  not  providing 

physical  care 
Fails  to  visit  or  inquire 
about  infant 

Wants  to  leave  hospital  AMA 
with  baby 

Obstructive  behavior  towards 
child's  medical  care 
Places  child  in  dangerous 
situation 

Threats  of  violent  behavior  or 
actual  violent  behavior 
towards  hospital  staff 
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Table  3. 

Reasons  for  Referrals  to  a  Child  Abuse  Registry 
(From  Ounsted,  Roberts,  Gordon,  et  al . ,  1982) 

Categories  of  Reasons  for  Referrals 

Doubt  about  parenting  ability  (attitude,  actual  harm  to 
infant,  indifference,  rejection,  display  of  temper) 

Psychiatric  history 

Behaviors  in  hospital 

Social  and  medical  problems  (housing,  income,  marriage) 
Previous  abuse  of  children 

Maternal  illness  or  handicap  (includes  low  IQ) 

Social  problems  alone 

Miscellaneous  (e.g.  adoption  considered) 
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Table  4 

A  Review  of  Selected  Past  Research  on  Risk 
Factors  for  Child  Maltreatment 

This  table  compares  which  studies  support  and  do  not 
support  the  listed  items  as  risk  factors  for  child  maltreat¬ 
ment. 


Item 

Supports 

Does  Not  Support 

Parental  Characteristics 

Poor  Childhood 

Altemeier  et  al., 

Nurture 

1982 

Monaghan , Gilmore , 
et  al. ,1986 

Oates  et  al.,1979 

Abused  as  a 

Egeland , 1988 

Widom, 1989 

Child 

Council  on  Sci . 

Aff . ,1985 

Hunter  et  al.,1978 

Young  Age 

Stier , 1989 

Zuravin, 1988 

U.S.  HHS ,1988 

Altemeier  et  al.,1982 

Race 

U.S.  HHS, 1988 

Gen.  Character 

Monaghan , Gilmore , 
et  al. ,1986 

Egeland, 1979 

Low  Intelligence/ 

Hunter  et  al.,1978 

Altemeier  et  al.,1982 

Education 

Seagull , Scheurer , 
1986 

Mental  Illness 

Council  on  Sci. 

Aff. ,1985 

Alcohol  Abuse 

Council  on  Sci. 

Aff. ,1985 

Altemeier  et  al.,1984 

Drug  Abuse 

Council  on  Sci. 

Aff. ,1985 

Altemeier  et  al.,1984 

Low  Self-Esteem 

Altemeier  et  al., 
1982 

Evans , 1980 

Aggressive 

Altemeier  et  al., 
1982 

Evans , 1980 

Violent 

Avison, Turner 

Noh ,1986 

Impulsive 

Hunter  et  al.,1978 

Depression/Apathy 

Evans ,1980 

Stern , 1973 

Hunter  et  al.,1978 

External  Locus 

Ellis , Milner , 1981 

of  Control 

Evans , 1980 

( continued) 
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_ Item _ 

Parenting  Style 
Unreasonable  Ex¬ 
pectations  of 
Child 


Very  Punitive 

Family  Profile 
Low  Income 


Recent  Job  Change 
/Loss 

Lack  of  Supports 


Social  Isolation 


Life  Stresses 


Chaotic  Lifestyle 


Poor  Housing 


City  vs.  Rural 
Spousal  Conflict 


Family  Violence 


Health  Problems 
2_4  Children  in 
Family 

Children  Spaced 
Closely  Together 
Inadeguate  Child 
Care 


Table  4 .  continued 

Supports 

Avison , Turner , 
Noh,1986 
Council  on  Sci. 

Aff . ,1985 
Oates  et  al.,1979 
Avison , Turner 
Noh ,1986 


U.S.  HHS ,1988 
Council  on  Sci. 
Aff. ,1985 

Hunter  et  al.,1978 
Oates  et  al.,1979 
Krugman  et  al., 
1986 

Hunter  et  al.,1978 

Oates  et  al.,1979 
Council  on  Sci. 

Aff. ,1985 
Stern, 1973 
Hunter  et  al.,1978 
Egeland , Breiten- 
bucher , Rosenberg , 
1980 

Egeland , Breiten- 
bucher , Rosenberg , 
1980 

Oates  et  al.,1979 
Evans , 1980 
Council  on  Sci. 

Aff. ,1985 
Oates  et  al.,1979 

Altemeier  et  al., 
1985 

Hunter  et  al.,1978 
Council  on  Sci. 

Aff. ,1985 
Widom, 1989 
Oates  et  al.,1979 
U.S.  HHS, 1988 
Zuravin, 1988 
Hunter  et  al.,1978 

Hunter  et  al.,1978 


Does  Not  Support 

Altemeier  et  al., 
1984 


Evans , 1980 


Daro, 1988 


Steinberg , Catalano , 
Dooley, 1981 
Altemeier  et  al., 
1984 

Altemeier  et  al., 
1984 


U.S.  HHS, 1988 


Zuravin , 1988 


( continued) 
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Table  4 .  continued 


_ Item _ Supports 

Family  Profile .  continued 
Children  Have  Zuravin,1988 

Different  Fathers 


Current  Pregnancy 
Unplanned/ 
Unwanted 


Poor  Prenatal 
Care 

Pregnancy/Delivery 

Complications 

Newborn 

Health 

Spent  Time  in  ICU 

Prematurity 

Low  Birth  Weight 


IUGR 

Behavior /Temper¬ 
ament 
Gender 

"Wrong"  Gender 
Twins 

Birth  Order  >2 


Altemeier  et  al . , 
1982 

Oates  et  al.,1979 
Zuravin , 1987 
Hunter  et  al.,1978 
Hunter  et  al.,1978 

Altemeier  et  al., 
1985 


Hergenroeder 
et  al . , 1985 


Hergenroeder 
et  al . , 1985 
Vietze  et  al.,1980 
Stern , 1973 


Egeland , 1979 
Vietze  et  al.,1980 
U.S.  HHS ,1988 
Hunter  et  al.,1978 
Groothuis  et  al., 
1982 

Hergenroeder 
et  al. ,1985 


In-hospital  Behaviors 


Mother  and  Baby 
Separated 
Poor  Parent-Child 
Interaction 

Rejection/ 

Hostility 

Poor  Parenting 
Skills 


Hergenroeder 
et  al. ,1985 
Egeland, 1979 
Vietze  et  al . , 
1980 

Avison, Turner , 

Noh ,1986 

Oates  et  al.,1979 
Avison , Turner , 
Noh, 1986 
Egeland ,1979 


Does  Not  Support 


Lynch, 1976 


Egeland , 1979 


Leventhal , Egerter , 
Murphy ,1984 
Leventhal , Egerter , 
Murphy, 1984 


Leventhal , Berg , 
Egerter , 1987 


Starr ,1987 


( continued) 
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_ Item 

Out-of -hospital 

More  Exposure 
Between  Parent 
and  Child 


Table  4 ,  continued 

_ Supports _ Does  Hot  Support 

Behaviors 

Altemeier  et  al., 

1982 


Table  5 


Experts'  2x2  Contingency  Table 
of  Paired  GRS  Ratings 


Expert  #2  GRS  Rating 


High/ 

Moderate 

Risk 

Low/No 

Risk 

Expert 

#1 

High/ 

Moderate 

Risk 

a=ll 

b=0 

f  =11 

1 

GRS 

Rating 

Low/ 

No 

Risk 

c=3 

d=l  8 

f  =21 

2 

n  =14 

1 

n  =18 

2 

N=32 

Percent 

agreement 

(High/Moderate  Risk)= 

2a  =88% 

n  +f 

1  1 


Percent  agreement  (Low/No  Risk1=  2d  =92% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Pq)=91% 

Kappa-2 ( ad-bc )  =0.80 

nf +nf 


Table  6 


Experts'  2x2  Contingency  Table 
of  Paired  CRS  Ratings 


Expert  #2  CRS  Rating 


Expert 

High 

#1 

Risk 

CRS 

Rating 

Low 

Risk 

High 

Risk 

Low 

Risk 

a=19 

b=l 

f  =20 

1 

c=4 

d=8 

f  =12 

2 

n  =23 

n  =9 

N=32 

1 

2 

Percent  agreement  (High/Moderate  Risk)=__2a_=88% 

n  +f 

1  1 

Percent  agreement  (Low/No  Risk)=__2d_=76% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Pq)=84% 

Kappa-2 ( ad-bc )  =0.65 

n  f  +n  f 
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Table  7 


Experts'  CRS  Item  By  Item  Agreements 
(Ranked  in  descending  order  by  kappa^,  then  by  P  ) 


CRS 

Item 

KaDDa 

P 

P 

21 

Cooperativeness-Mother 

*-  L  w 

1.00 

o 

1.00 

e’ 

0.98 

19 

Care  of  Newborn  by  Mother 

1.00 

1.00 

0.97 

4 

Illicit  Drug  Use-Mother 

0.92 

0.98 

0.69 

14 

Housing 

0.92 

0.98 

0.76 

7 

Criminal  Record-Mother 

0.88 

0.98 

0.83 

17 

Attitude 

0.84 

0.96 

0.75 

15 

Provisions  for  Baby 

0.80 

0.95 

0.77 

9 

Care  as  a  Child-Mother 

0.74 

0.93 

0.75 

5 

Alcohol  Use-Mother 

0.73 

0.92 

0.70 

11 

General  Character-Father 

0.71 

0.88 

0.58 

18 

Prenatal  Care 

0.69 

0.88 

0.62 

1 

Care  of  Older  Sibs 

0.62 

0.92 

0.80 

12 

Family  Conflict 

0.62 

0.86 

0.61 

8 

History  of  Violence-Mother 

0.61 

0.90 

0.74 

13 

Stability  of  Couple 

0.58 

0.85 

0.64 

2 

Intelligence-Mother 

0.47 

0.97 

0.95 

6 

Motivation-Mother 

0.44 

0.85 

0.72 

10 

Age-Mother 

0.43 

0.80 

0.65 

16 

Social  Supports 

0.40 

0.89 

0.82 

3 

Psychiatric  History-Mother 

0.32 

0.86 

0.80 

20 

Visiting  with  Baby 

0.15 

0.86 

0.84 

22 

Danger  to  Child  in  Hospital 

* 

1.0 

* 

*  100%  agreement,  all  in  cell  "a"  of  contingency  table;  Pe 
(percent  expected  agreement)  and  kappa  not  computable 


Table  8 


Experts'  2x2  Contingency  Table 
of  CRS  and  GRS  Ratings 


Experts'  GRS  Rating 


High/ 

Moderate 

Risk 

Low/No 

Risk 

> 

Same  High  a=25 

b=19 

f  =44 

1 

Expert's  Risk 

CRS 

Rating  Low  c=Q 

d=20 

f  =20 

2 

Risk 

n  =25 

n  =39 

N=64 

1 

2 

Sensitivity=100% 

Specif icity=51% 

Percent  agreement  (High/Moderate  Risk^  2a  =72% 

n  +f 

1  1 

Percent  agreement  (Low/No  Risk)=  2d  =68% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Pq)=70% 

Kapoa=2 ( ad-bc )  =0.45 

n  f  +n  f 


Table  9 


Nurses'  GRS  Ratings  vs. 
Experts'  Consensus  GRS  Ratings 


Experts'  Consensus  GRS  Rating 


Nurses ' 
GRS 

Rating 


High/ 

Moderate 

Risk 

1 

Low/No 

Risk 

High/ 

Moderate 

Risk 

a=4 

b=0 

f  =4 

1 

Low/ 

No 

Risk 

c=6 

d=10 

f  =16 

2 

n  =10 

1 

n  =10 

2 

N=20 

Sensitivity=40% 

Specif icity=l 00% 

Percent  agreement  (High/Moderate  Risk)=  2a  =57% 

n  +f 

1  1 

Percent  agreement  (Low/No  Risk1=  2d  =77% 

n  +f 

2  2 

Percent  overall  observed  agreement  (P  )=70% 

o 

Kappa=2 ( ad-bc )  =0.40 

n  f  +n  f 
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Table  10 

Nurses'  CRS  Ratings  vs. 
Experts'  Consensus  GRS  Ratings 


Experts'  Consensus  GRS  Rating 


High 

Nurses'  Risk 
CRS 

Rating  Low 
Risk 


High/ 

Moderate 

Risk 

Low/No 

Risk 

a=7 

b=2 

f  =9 

1 

C=4 

d=7 

f  =11 

2 

n  =11 

n  =9 

N=20 

1 

2 

Sensitivity=64% 

Specif icity=7 8% 

Percent  agreement  (High/Moderate  Risk~)  =  2a  =70% 

n  +f 

1  1 

Percent  agreement  (Low/No  Risk1=  2d  =70% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Pq)=70% 

Kappa=2 ( ad-bc )  =0.41 

n  f  +n  f 

12  2  1 


Table  11 


Pediatricians'  GRS  Ratings  vs. 
Experts'  Consensus  GRS  Ratings 


Experts'  Consensus  GRS  Rating 


High/ 

Moderate 

Risk 

Low/No 

Risk 

Pedia¬ 

tricians' 

High/ 

Moderate 

Risk 

a=5 

b=l 

f  =6 

1 

GRS 

Rating 

Low/ 

No 

Risk 

c=3 

d=ll 

f  =14 

2 

n  =8 

1 

n  =12 

2 

N=20 

Sensitivity=63% 

Specif icity=9 2% 

Percent  agreement  (High/Moderate  Ris]Q=  2a  =71% 

n  +f 

1  1 

Percent  agreement  (Low/No  Risk)=  2d  =85% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Pq)=80% 

Kappa=2  f  ad-bc )  =0.57 

n  f  +n  f 


Table  12 


Pediatricians'  CRS  Ratings  vs. 
Experts'  Consensus  GRS  Ratings 


Experts'  Consensus  GRS  Rating 


High/ 

Moderate 

Risk 

Low/No 

Risk 

Pedia-  High  a=5 

b=l 

f  =6 

1 

tricians'  Risk 

CRS 

Rating  Low  c=3 

d=ll 

f  =14 

2 

Risk 

n  =8 

n  =12 

N=2  0 

1 

2 

Sensitivity=63% 

Specif icity=9 2% 

Percent  agreement  (High/Moderate  Risk)=__2a__=71% 

n  +f 

1  1 

Percent  agreement  (Low/No  Risk ) =  2d  =85% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Pq)=80% 

Kappa=2 ( ad-bc )  =0.57 

n  f  +n  f 


Table  .13 


Highest  Non-expert  GRS  Ratings  vs. 
Experts'  Consensus  GRS  Ratings 


Experts'  Consensus  GRS  Rating 


High/ 

Moderate 

Risk 

Low/No 

Risk 

Highest 

Non-expert 

High/ 

Moderate 

Risk 

a=7 

b=2 

f  =9 

1 

GRS 

Rating 

Low/ 

No 

Risk 

c=6 

d=12 

f  =18 

2 

n  =13 

1 

n  =14 

2 

N=27 

Sensitivity=54% 

Specif icity=8 6% 

Percent  agreement  (High/Moderate  Ris30=  2a  =64% 

n  +f 

1  1 


Percent  agreement  (Low/No  Risk)=__2d__=75% 

n  +f  2 

Percent  agreement  (Low/No  Risk) =75!  2 

Percent  overall  observed  agreement  (Pq)=70% 

Kappa=2 ( ad-bc )  =0.40 

n  f  +n  f 


Table  14 


Highest  Non-expert  CRS  Ratings  vs. 
Experts'  Consensus  GRS  Ratings 


Experts'  Consensus  GRS  Rating 


High/ 

Moderate 

Risk 

i  ( 

Low/No 

Risk 

Highest  High  a=10 

b=3 

f  =13 

1 

Non-expert  Risk 

CRS 

Rating  Low  c=3 

d=ll 

f  =14 

2 

Risk 

n  =13 

n  =14 

N=27 

1 

2 

Sensitivity=77% 

Specif icity=7 9% 

Percent  agreement  (High/Moderate  Risk ) =  2a  =77% 

n  +f 
1  1 


Percent  agreement  (Low/No  Risk)=__2d___=79% 

n  +f 

2  2 

Percent  overall  observed  agreement  (Po)=78% 

Kappa=2 ( ad-bct  =0.55 
nf+nf 
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Table  15 

Breakdown  of  Evaluations  By  Clinician  Type 


No  Evaluations  Returned 

26  (7%) 

At  Least  One  Evaluation  Returned 

337  (93%) 

Only  a  Nurse's  Evaluation 

41  (11%) 

Only  a  Pediatrician's 

Evaluation 

44  (12%) 

Only  a  Social  Worker's 

Evaluation 

15  (4%) 

Both  a  Nurse's  and  a 

Pediatrician's  Evaluation 

133  (37%) 

Both  a  Nurse's  and  a  Social 

Worker's  Evaluation 

28  (8%) 

Both  a  Pediatrician's  and  a 

Social  Worker's  Evaluation 

19  (5%) 

Nurse's,  Pediatrician's  and 

Social  Worker's  Evaluation 

57  (16%) 

Total  Evaluations  by 

Nurses 

259  (71%) 

Total  Evaluations  by 

Pediatricians 

253  (70%) 

Total  Evaluations  by 

Social  Workers 

119  (33%) 
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Table  16 

Demographic  Profile 


Entire  Study  Those  with  Those  without 

Group  Evaluations  Evaluations 


( N=363 

( N=3  37 ) 

( N=26 1 

Race 

Black 

67% 

67% 

63% 

White 

17% 

18% 

8% 

Hispanic 

16% 

15% 

21% 

Oriental 

1% 

0% 

8% 

Religion 

None/Unknown 

13% 

13% 

11% 

Catholic 

33% 

32% 

44% 

Protestant 

49% 

51% 

22% 

Other 

5% 

4% 

22% 

Marital  Status 

Single 

79% 

79% 

76% 

Married 

12% 

12% 

20% 

Separated 

5% 

6% 

0% 

Divorced 

Number  of  Prior 

3% 

Pregnancies 

3% 

4% 

None 

22% 

20% 

40% 

One  or  more 

78% 

80% 

60% 

Five  or  more 

Number  of  Prior 

28% 

Deliveries 

28% 

28% 

None 

32% 

31% 

44% 

One  or  more 

68% 

69% 

56% 

Five  or  more  4% 

Current  Gestation 

4% 

0% 

Premature 

22% 

21% 

24% 

Term 

74% 

74% 

72% 

Post-dates  5% 

Number  of  Babies 

5% 

4% 

Singleton 

96% 

96% 

92% 

Twins 

4% 

4% 

8% 

Delivery 

Planned  C-sec 

6% 

6% 

8% 

Crash  C-sec 

3% 

3% 

0% 

NSVD 

77% 

77% 

76% 

Induced  vag. 

9% 

9% 

12% 

C-sec  after  5% 

trial  of  labor 

5% 

4% 

Gender  of  Baby 

Male 

49% 

49% 

52% 

Female 

51% 

51% 

48% 

Feeding 

Breast 

11% 

12% 

0% 

Bottle 

84% 

84% 

80% 

Both  5% 

Hospital  Billing 

4% 

20% 

Self-Pay 

5% 

5% 

5% 

Insurance 

14% 

13% 

26% 

Public  Aid 

82% 

82% 

68% 
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Table  17 

Risk  Rating  Assignments  By  Clinician  Type 
Clinician  Type _ N _ No.  High  Risk _ %  High  Risk 


Nurse 

259 

82 

32 

Pediatrician 

253 

91 

36 

Social  Worker 

119 

99 

83 

Highest  CRS  Rating 

337 

168 

50 

CRS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 
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Table  18 

Percentage  of  Times  Each  CRS  Item 
Was  Unknown  By  Clinician  Type 

Item  Number  Expert  Nurse  Pediatrician  Social  Worker 


8.0 

67.0 

52.6 

34.9 

1.6 

13.9 

8.3 

3.4 

4.7 

64.1 

61.7 

24.4 

0 

44.0 

24.9 

8.4 

0 

57.9 

35.2 

17.6 

3 . 1 

13 . 1 

26 . 1 

2.5 

4.7 

80.3 

76.3 

26.1 

14 . 1 

91.9 

95.3 

77.3 

3 . 1 

78.0 

85.8 

50 . 4 

7.8 

18 . 5 

17.8 

5.0 

4.7 

46.7 

51.8 

17.6 

4.7 

67.2 

64.4 

13.4 

6.3 

52.5 

62 . 1 

26.1 

3 . 1 

48 . 3 

42 . 7 

2.5 

1.6 

25.5 

23 . 3 

2.5 

3.1 

17.8 

24 . 1 

1.7 

4.7 

43.2 

53.0 

11.8 

4.7 

34.0 

25.3 

9.2 

17.2 

9.3 

7.1 

5.9 

23.4 

22.0 

13.4 

18.5 

1.6 

3.5 

2.4 

1.7 

9.4 

13.9 

10.7 

5.0 

N=64 

N=259 

N=253 

N=119 
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Table  19 


Items 

Ranked 

by  Percentage 

Unknown 

By  Each 

Clinician  Type 

Expert 
Item  # 

20 

% 

23.4 

Nurse 
Item  # 

8 

% 

91.9 

Pediatrician 
Item  it  % 

8  95.3 

Social 
Item  it 

8 

Worker 

% 

77.3 

19 

17.2 

7 

80.3 

9 

85.8 

9 

50.4 

8 

14.1 

9 

78.0 

7 

76.3 

1 

34.9 

22 

9.4 

12 

67.2 

12 

64.4 

7 

26.1 

1 

8.0 

1 

67.0 

13 

62.1 

13 

26.1 

10 

7.8 

3 

64.1 

3 

61.7 

3 

24.4 

13 

6.3 

5 

57.9 

17 

53.0 

20 

18.5 

3 

4.7 

13 

52.5 

1 

52.6 

5 

17.6 

7 

4.7 

14 

48.3 

11 

51 . 8 

11 

17.6 

11 

4.7 

11 

46.7 

14 

42.7 

12 

13.4 

12 

4.7 

4 

44.0 

5 

35.2 

17 

11.8 

17 

4.7 

17 

43 . 2 

6 

26.1 

18 

9.2 

18 

4.7 

18 

34.0 

18 

25.3 

4 

8.4 

6 

3.1 

15 

25.5 

4 

24.9 

19 

5.9 

9 

3.1 

20 

22.0 

16 

24.1 

10 

5.0 

14 

3.1 

10 

18.5 

15 

23 . 3 

22 

5.0 

16 

3.1 

16 

17.8 

10 

17.8 

2 

3.4 

2 

1.6 

2 

13.9 

20 

13.4 

6 

2.5 

15 

1.6 

22 

13.9 

22 

10.7 

14 

2.5 

21 

1.6 

6 

13.1 

2 

8.3 

15 

2.5 

4 

0 

19 

9.3 

19 

7.1 

16 

1.7 

5 

0 

21 

3.5 

21 

2.4 

21 

1.7 

N=64 


N=259 


N=253 


N=119 
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Table  20 

Items  Ranked  By  Percentage  Unknown: 

Only  Where  Nurse  and  Pediatric  Evaluations  Both  Returned 

Nurse  Pediatrician 


Item  # 

8 

% 

90.2 

Item  # 

8 

% 

98.5 

7 

75.2 

9 

87.2 

9 

72.9 

7 

77.4 

1 

64.0 

13 

65.4 

3 

60.9 

12 

63 . 2 

12 

60.9 

3 

60.9 

5 

55.6 

11 

55.6 

13 

54 . 1 

17 

53.4 

11 

48.9 

1 

50.6 

4 

48 . 1 

14 

42.1 

14 

45.1 

5 

36.8 

17 

41.4 

6 

32 . 3 

18 

40.6 

4 

30.1 

20 

27.1 

18 

26 . 3 

15 

25.6 

16 

23 . 3 

10 

17 . 3 

15 

18.0 

16 

15.0 

10 

15.0 

2 

13.5 

20 

10.5 

6 

10.5 

2 

8.3 

22 

8.3 

22 

7.5 

19 

3 . 8 

19 

5.3 

21 

2 . 3 

21 

0.8 

N=1 90 


N=190 
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Table  21 

Nurse-Pediatrician  CRS  Item  By  Item  Agreements  (N=190) 
(Ranked  in  descending  order  by  kappa^,  then  by  Pq) 


CRS  Item  Number 

Kappa^ 

P 

P 

4* 

0.76 

0.91 

0.62 

3 

0.75 

0.97 

0.89 

1* 

0.62 

0.88 

0.67 

12* 

0.55 

0.95 

0.89 

11 

0.50 

0.81 

0.62 

5* 

0.47 

0.87 

0.76 

10* 

0.42 

0.81 

0.68 

17 

0.40 

0.89 

0 . 82 

18 

0.39 

0.77 

0.61 

13 

0.35 

0.86 

0.78 

19 

0.29 

0.95 

0.93 

6 

0.29 

0.79 

0 . 71 

15 

0.25 

0.91 

0.88 

14 

0.21 

0.89 

0.86 

20 

0.20 

0.83 

0.79 

2 

0.18 

0.88 

0.86 

21 

0.13 

0.97 

0.97 

22 

0.06 

0.96 

0.96 

9 

0 

0.89 

0.89 

8 

0 

0.80 

0.80 

7 

- 

- 

- 

16 

-0.06 

0.86 

0.87 

*  Designates  items  with 

fair  agreement  on  Tables  21 , 

22,  and 

P  =Percent  expected  agreement 

e 
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Table  22 

Pediatrician-Social  Worker  CRS  Item  By  Item  Agreements 

( N=7  6 ) 

(Ranked  in  descending  order  by  kappa^,  then  by  Po) 


CRS  Item  Number 

Kaona 

P 

P 

4* 

0.73 

0.88 

0.57 

5* 

0.68 

0.86 

0.57 

21 

0.66 

0.97 

0.91 

1* 

0.65 

0.84 

0.53 

19 

0.64 

0.92 

0.78 

11 

0.53 

0.80 

0.56 

12* 

0.52 

0.84 

0.67 

10* 

0.48 

0.83 

0.67 

18 

0.48 

0.83 

0.68 

3 

0.43 

0.88 

0.79 

20 

0.32 

0.80 

0.70 

13 

0.30 

0.79 

0.70 

15 

0.30 

0.78 

0.68 

2 

0.27 

0.88 

0.84 

6 

0.27 

0.75 

0.65 

14 

0.24 

0.84 

0.79 

17 

0.22 

0.86 

0.82 

9 

0.22 

0.79 

0.73 

16 

0.20 

0.80 

0.75 

22 

0.08 

0.91 

0 . 90 

7 

0 

0.96 

0.96 

8 

-0.25 

0.67 

0 .73 

*  Designates  items  with  fair  agreement  on  Tables  21,  22,  and  23 
Pe=Percent  expected  agreement 


Table  23 


Nurse-Social  Worker  CRS  Item  By  Item  Agreements  (N=85) 
(Ranked  in  descending  order  by  kappa^,  then  by  Pq) 


CRS  Item  Number 

KaDDa 

P 

P 

8 

L  L  w 

1.00 

1.00 

0.80 

10* 

0.77 

0.92 

0.63 

4* 

0.61 

0.82 

0.54 

1* 

0.60 

0.83 

0.57 

9 

0.55 

0.88 

0.74 

14 

0.51 

0.91 

0 . 82 

22 

0.48 

0.95 

0.90 

5* 

0.47 

0.80 

0.63 

12* 

0.42 

0.88 

0 . 80 

20 

0.37 

0.81 

0.69 

6 

0.33 

0.77 

0.66 

11 

0.27 

0.71 

0.60 

18 

0.26 

0.73 

0.64 

15 

0.23 

0.78 

0.71 

19 

0.20 

0.87 

0.84 

21 

0.17 

0.92 

0.91 

16 

0.17 

0.85 

0.82 

2 

0.14 

0.82 

0.79 

17 

0.09 

0.84 

0.82 

7 

0 

0.95 

0.95 

3 

-0.08 

0.84 

0.86 

13 

-0.12 

0.70 

0.73 

*  Designates  items  with 

fair  agreement  on  Tables  21, 

22,  and 

P  =Percent  expected  agreement 

e 
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